new to octave, confused about load(), dlmread et cetera

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

new to octave, confused about load(), dlmread et cetera

justin.cress
this seems to be the most asked question, but before I ask it, i'd like to assure everyone i've been googling around for a couple of hours now and haven't figured out where i'm going wrong,

What i'd like to be able to do is load a bunch of column vectors from a csv or text file (created by stata) and have octave grab the headers as variable names.  I've tried all the functions I could find documentation on, but none of them work... csvread ignores headers, dlmread is the same... and load()  gives

error: load: failed to read matrix from file `ps1data.csv'
error: called from:
error:   readtest.m at line 9, column 1

error: load: failed to read matrix from file `ps1data.txt'
error: called from:
error:   readtest.m at line 9, column 1

for csv and tab delimit, respectively.  

any help would be appreeciated.


Reply | Threaded
Open this post in threaded view
|

Re: new to octave, confused about load(), dlmread et cetera

forkandwait
justin.cress <boywonder410 <at> gmail.com> writes:


> What i'd like to be able to do is load a bunch of column vectors from a csv
> or text file (created by stata) and have octave grab the headers as variable
> names.  

A matrix doesn't have header names, hence dlmread etc won't work with headers.
There is nothing with header names native to matlab/ octave, really.  I would
try to work with the matrix without header names, just remember what each column
means.  

What you want (if you want it to look like a statistics language) is to convert
each column into a struct field, and have a "struct array".  The problem is that
 very few functions work with this data structure, and you will have to convert
it into a matrix in order to do tabulations or whatever.  I don't think there is
a function to read in data like this either.

My hunch is that you are trying to make octave/ matlab behave like
stata/R/sas/etc, and octave doesn't really do that well.

That may not be much help, but maybe if share the particular application someone
can help you do it more idiomatically (idiomatic to octave, anyway).

_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: new to octave, confused about load(), dlmread et cetera

Jaroslav Hajek-2
In reply to this post by justin.cress
On Tue, Aug 31, 2010 at 3:04 AM, justin.cress <[hidden email]> wrote:

>
> this seems to be the most asked question, but before I ask it, i'd like to
> assure everyone i've been googling around for a couple of hours now and
> haven't figured out where i'm going wrong,
>
> What i'd like to be able to do is load a bunch of column vectors from a csv
> or text file (created by stata) and have octave grab the headers as variable
> names.  I've tried all the functions I could find documentation on, but none
> of them work... csvread ignores headers, dlmread is the same... and load()
> gives
>
> error: load: failed to read matrix from file `ps1data.csv'
> error: called from:
> error:   readtest.m at line 9, column 1
>
> error: load: failed to read matrix from file `ps1data.txt'
> error: called from:
> error:   readtest.m at line 9, column 1
>
> for csv and tab delimit, respectively.
>
> any help would be appreeciated.
>
>

The reason why there is no "just load it" function in Octave for this
kind of data is that there is no built-in datatype for matrices with
named (and possibly heterogeneous) columns. Octave's arrays are, in
general, multidimensional and homogeneous (except cell and struct
arrays), and carry no row or column labels.

--
RNDr. Jaroslav Hajek, PhD
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz

_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: new to octave, confused about load(), dlmread et cetera

CdeMills
Jaroslav Hajek-2 wrote
The reason why there is no "just load it" function in Octave for this
kind of data is that there is no built-in datatype for matrices with
named (and possibly heterogeneous) columns. Octave's arrays are, in
general, multidimensional and homogeneous (except cell and struct
arrays), and carry no row or column labels.
If you are ready to deal with occasional bugs, I released a few days ago a package on octave-forge whose name is 'dataframe', with preciselly the goal of having matrix-like structures with named rows and columns, and the abilitity to select by name. It can also read  CSV files, infer data types, rows and columns names. To try it:
go to http://octave.sourceforge.net/packages.php
download dataframe-0.4.tar.gz
from octave:
pkg install '~/the/path/with/dataframe-0.4.tar.gz'
pkg load dataframe
test and enjoy

Regards

Pascal
Reply | Threaded
Open this post in threaded view
|

Re: new to octave, confused about load(), dlmread et cetera

forkandwait
In reply to this post by justin.cress

> What i'd like to be able to do is load a bunch of column vectors from a csv
> or text file (created by stata) and have octave grab the headers as variable
> names.  

You might also try using "textread" (help textread).  With textread you can
snarf columns into particular variables.

_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: new to octave, confused about load(), dlmread et cetera

andy buckle
In reply to this post by justin.cress
> this seems to be the most asked question, but before I ask it, i'd like to
> assure everyone i've been googling around for a couple of hours now and
> haven't figured out where i'm going wrong,
>
> What i'd like to be able to do is load a bunch of column vectors from a csv
> or text file (created by stata) and have octave grab the headers as variable
> names.

Another idea!

If you have a lot of control (I have no idea what stata can do) you
may be able to get stata to automatically write an octave script to
create your variables and headers.

--
/* andy buckle */
_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: new to octave, confused about load(), dlmread et cetera

justin.cress
In reply to this post by forkandwait
thanks for all the responses everyone, and, I think the general theme is correct, I was trying to use octave too much like stata... 

So, if I don't know exactly which order the columns are in (in the data) is there an easy way to let 

Y = load ( .. args ..)    be a Y var  

and grab a matrix of X vars like 

X = load ( ... args ..)    

I guess my question is, if i just wanted to do basic OLS (or something similar) in octave, what's the general approach?

On Tue, Aug 31, 2010 at 2:15 AM, forkandwait <[hidden email]> wrote:
justin.cress <boywonder410 <at> gmail.com> writes:


> What i'd like to be able to do is load a bunch of column vectors from a csv
> or text file (created by stata) and have octave grab the headers as variable
> names.

A matrix doesn't have header names, hence dlmread etc won't work with headers.
There is nothing with header names native to matlab/ octave, really.  I would
try to work with the matrix without header names, just remember what each column
means.

What you want (if you want it to look like a statistics language) is to convert
each column into a struct field, and have a "struct array".  The problem is that
 very few functions work with this data structure, and you will have to convert
it into a matrix in order to do tabulations or whatever.  I don't think there is
a function to read in data like this either.

My hunch is that you are trying to make octave/ matlab behave like
stata/R/sas/etc, and octave doesn't really do that well.

That may not be much help, but maybe if share the particular application someone
can help you do it more idiomatically (idiomatic to octave, anyway).

_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave


_______________________________________________
Help-octave mailing list
[hidden email]
https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: new to octave, confused about load(), dlmread et cetera

CdeMills
justin.cress wrote
thanks for all the responses everyone, and, I think the general theme is
correct, I was trying to use octave too much like stata...

So, if I don't know exactly which order the columns are in (in the data) is
there an easy way to let

Y = load ( .. args ..)    be a Y var

and grab a matrix of X vars like

X = load ( ... args ..)

I guess my question is, if i just wanted to do basic OLS (or something
similar) in octave, what's the general approach?
Octave is mostly used by engineers, so is more oriented towards numerical computations, linear algebra, signal processing and the like.

Regarding the problem of identifying columns from their name, this is the purpose of the 'dataframe' package I mentionned, which is similar to R data.frame.

About OLS, let say that A is your design matrix, X your unknown and B your observations,
you have to solve Ax = B
which can be computed as x = A\B
The '\' is a specific operator, and can detect cases of undetermined / badly scaled systems.

Regards

Pascal