Correlation with nan package

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Correlation with nan package

shivax
hi,
i try to study correlation

i've the vector A and B

a.txt <http://octave.1599824.n4.nabble.com/file/t372898/a.txt>  

b.txt <http://octave.1599824.n4.nabble.com/file/t372898/b.txt>  

i use corrcoef of package nan

>> pkg load naan
>> corrcoef(a,b)

i get this error:

>> corrcoef(a,b)
error: out of memory or dimension too large for Octave's index type
error: called from
    corrcoef at line 221 column 28
>>








--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Correlation with nan package

Alois Schloegl-7
corrcoef expects column vectors. So any  of the following commands might
fix the problem for you:

  corrcoef(a',b')

     corrcoef(a(:),b(:))



Cheers,
   Alois



On 2018-06-24 15:50, shivax wrote:

> hi,
> i try to study correlation
>
> i've the vector A and B
>
> a.txt <http://octave.1599824.n4.nabble.com/file/t372898/a.txt>  
>
> b.txt <http://octave.1599824.n4.nabble.com/file/t372898/b.txt>  
>
> i use corrcoef of package nan
>
>>> pkg load naan
>>> corrcoef(a,b)
> i get this error:
>
>>> corrcoef(a,b)
> error: out of memory or dimension too large for Octave's index type
> error: called from
>     corrcoef at line 221 column 28
>
>
>
>
>
>
>
> --
> Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Correlation with nan package

shivax
hi all,

i'm using this package

i use corrcoef(a,b)

but sometimes i see this return: Inf (infinite)


how it is possible to return Inf if correlation varies from a minimum of 1
to -1?








--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Correlation with nan package

Alois Schloegl-7
Hi,


currently, corrcoef from the nan-toolbox uses the faster one-pass 
algorithm, that is known to have reduced accuracy.
It's an issue when the data has a very low coefficient of variation
(i.e. std(x)/mean(x) is in the order of sqrt(eps) ) . (for details see
also [1]).

There is a simple workaround, just remove the mean before doing corrcoef.
It will produce the same correlation, and is not subject to the limited
accuracy.
You can do this e.g. with:
    corrcoef(center(a),center(b))

This approach corresponds to the more accurate two-pass algorithm.
Eventually, I might implement the more accurate two-pass algorithm.
Because it has some implications in terms of speed and memory
requirement, users might want to control whether one or the other
algorithm is used.

I hope this answers your questions,

Cheers,
  Alois
 
 


[1]
https://stats.stackexchange.com/questions/94056/instability-of-one-pass-algorithm-for-correlation-coefficient

 


On 2018-07-15 02:01, shivax wrote:

> hi all,
>
> i'm using this package
>
> i use corrcoef(a,b)
>
> but sometimes i see this return: Inf (infinite)
>
>
> how it is possible to return Inf if correlation varies from a minimum of 1
> to -1?
>
>
>
>
>
>
>
>
> --
> Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html
>
>