Reading/Manipulating a csv file with columns that have different data types

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Reading/Manipulating a csv file with columns that have different data types

frobeniustaco
Hi folks!

----------

First off, here is a small sample of the data I'm working with:

1/7/2015,0,Day,39,8346,32,10,5,0
1/8/2015,0,Day,39,4204,32,7,3,0
1/9/2015,0,Day,39,6848,29,6,6,0

----------

Here's what I want to do with it, once I get it read into Octave:

Columns 1, 3, and 4:  I'll want to sort by month, shift (there will b 3 different strings as values) and machine id (also 3 different values).

Columns 5-8:  I'll want to do some calculations with these:

----------

I have been searching far and wide for a method of getting my data into octave in such a way that I can manipulate it as described above.  I have been able to at least read it a couple different ways (csvread, csv2cell, etc) but if I try to do any arithmetic on columns 5-8 I get errors.

Anyone have an idea how I can get the data setup so I can reach those objectives?   The most frustrating part of all this I have a pretty confident vision of how the rest of my script will look but I can't for the life of me figure out the upload (which seems like it should just be a few lines, at best).

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Reading/Manipulating a csv file with columns that have different data types

Thomas D. Dean-2
On 04/21/15 08:13, frobeniustaco wrote:

> Hi folks!
>
> ----------
>
> First off, here is a small sample of the data I'm working with:
>
> 1/7/2015,0,Day,39,8346,32,10,5,0
> 1/8/2015,0,Day,39,4204,32,7,3,0
> 1/9/2015,0,Day,39,6848,29,6,6,0
>
> ----------
>
> Here's what I want to do with it, once I get it read into Octave:
>
> Columns 1, 3, and 4:  I'll want to sort by month, shift (there will b 3
> different strings as values) and machine id (also 3 different values).
octave:40> pkg load io
octave:41> Z=csv2cell('xx.dat')
octave:42> A=cell2mat(Z(1:3,6:8))

Tom Dean

_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Reading/Manipulating a csv file with columns that have different data types

bpabbott
Administrator
In reply to this post by frobeniustaco

> On Apr 21, 2015, at 11:13 AM, frobeniustaco <[hidden email]> wrote:
>
> Hi folks!
>
> ----------
>
> First off, here is a small sample of the data I'm working with:
>
> 1/7/2015,0,Day,39,8346,32,10,5,0
> 1/8/2015,0,Day,39,4204,32,7,3,0
> 1/9/2015,0,Day,39,6848,29,6,6,0
>
> ----------
>
> Here's what I want to do with it, once I get it read into Octave:
>
> Columns 1, 3, and 4:  I'll want to sort by month, shift (there will b 3
> different strings as values) and machine id (also 3 different values).
>
> Columns 5-8:  I'll want to do some calculations with these:
>
> ----------
>
> I have been searching far and wide for a method of getting my data into
> octave in such a way that I can manipulate it as described above.  I have
> been able to at least read it a couple different ways (csvread, csv2cell,
> etc) but if I try to do any arithmetic on columns 5-8 I get errors.
>
> Anyone have an idea how I can get the data setup so I can reach those
> objectives?   The most frustrating part of all this I have a pretty
> confident vision of how the rest of my script will look but I can't for the
> life of me figure out the upload (which seems like it should just be a few
> lines, at best).
>
> Thanks!

The short script below converts your csv-file into a matrix of doubles.

csvfile = "test.txt";  
str = strsplit ((fileread (csvfile)), "\n");
n = cellfun (@isempty, str, "uniformoutput", true);
str = str(!n);
str = cellfun (@(s) strsplit (s, {",", "/"}), str, "UniformOutput", false);
r = numel (str);
str = [str{:}];
str = reshape (str, numel (str) / r, r);
# Replace shift text with numbers
str = strrep (str, "Day", "1");
str = strrep (str, "Second", "2");
str = strrep (str, "Night", "3");
# Convert all char data to double
str = cellfun (@str2double, str, "UniformOutput", true);

The result is

str =

      1      1      1
      7      8      9
   2015   2015   2015
      0      0      0
      1      1      1
     39     39     39
   8346   4204   6848
     32     32     29
     10      7      6
      5      3      6
      0      0      0

Ben


_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave