Read file containing multiple data types

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Read file containing multiple data types

Saji Kuttan
Hi,

I have a text file which is attached. The file has some header lines, that have to be skipped. I am trying to read the data in the file with different choices such as load, importdata, and fscanf. I think only fscanf can be used here. Even with fscanf, I have some problems.

I used fscanf as :

for i=1:33
[d,c1,c2,t]=fscanf(fid," %d %s %s %f", "C");
printf("%d %f\n", d,t)
x=fgetl(fid);  % to skip remaining part of the line
end


This code is not perfect because:

(1) I have to manually count the number of lines in the text file. I don't know to apply automatic EOF.
(2) There are some missing data, given as "..." at the end. Here the numeric field is changed to character. How it can be solved?

So please tell me how to read this file and save the data into different variables.



--
Saji P K
Assistant Professor
Department of Physical Oceanography
School of Marine Sciences,
Cochin University of Science and Technology,
Fine Arts Avenue
Kochi - 682016

Phone : (O) 0484 2863206 (Mob) 9744000804
Email : [hidden email]
Web : http://dpo.cusat.ac.in/faculty/pks/index.html


Reply | Threaded
Open this post in threaded view
|

Re: Read file containing multiple data types

Thomas D. Dean-2
On 11/3/18 3:08 AM, Saji Kuttan wrote:

> Hi,
>
> I have a text file which is attached. The file has some header lines,
> that have to be skipped. I am trying to read the data in the file with
> different choices such as load, importdata, and fscanf. I think only
> fscanf can be used here. Even with fscanf, I have some problems.
>
> I used fscanf as :
> *
> *
> *for i=1:33
> [d,c1,c2,t]=fscanf(fid," %d %s %s %f", "C");
> printf("%d %f\n", d,t)
> x=fgetl(fid);  % to skip remaining part of the line
> end
> *
> *
> *
> **
> This code is not perfect because:
>
> (1) I have to manually count the number of lines in the text file. I
> don't know to apply automatic EOF.
> (2) There are some missing data, given as "..." at the end. Here the
> numeric field is changed to character. How it can be solved?
>
> So please tell me how to read this file and save the data into different
> variables.
>

Since you have a mixture of data, it may be better to use fgetl and
process each line.

Write a script to read the file and print each line using the while
statement.  Then, add conditions to handle each of the types of lines.

The labeled lines at the top of the file can be detected and processed
with the if statement.

The lines containing only 59.5E and 60 look like stray entries.  But,
they can also be handled with the if statement.

The data lines are the else or fall through position.  You can use the
sscanf function and process them based on the number of elements scanned
and any error returned.

Tom Dean


Reply | Threaded
Open this post in threaded view
|

Re: Read file containing multiple data types

PhilipNienhuis
In reply to this post by Saji Kuttan
Saji Kuttan wrote

> Hi,
>
> I have a text file which is attached. The file has some header lines, that
> have to be skipped. I am trying to read the data in the file with
> different
> choices such as load, importdata, and fscanf. I think only fscanf can be
> used here. Even with fscanf, I have some problems.
>
> I used fscanf as :
>
> *for i=1:33[d,c1,c2,t]=fscanf(fid," %d %s %s %f", "C");printf("%d %f\n",
> d,t)x=fgetl(fid);  % to skip remaining part of the lineend*
>
> This code is not perfect because:
>
> (1) I have to manually count the number of lines in the text file. I don't
> know to apply automatic EOF.
> (2) There are some missing data, given as "..." at the end. Here the
> numeric field is changed to character. How it can be solved?
>
> So please tell me how to read this file and save the data into different
> variables.

This looks like the kind of datafiles I often get, textscan should be
perfect for this kind of jobs.

I first tried with:

   C = textscan (fid, "%d / %d: %s", "headerlines", 10)

but apparently the headerlines option didn't work here (could be due to line
endings, or maybe it's a bug).

So including a workaround for that headerlines issue your file can be read
conveniently with:

  fid = fopen (("data.txt", "r");
  ##  Workaround for non-functional "headerlines" option in textscan
  for ii=1:10
    fgetl (fid);
  endfor

  ## Remember file position when experimenting with textscan options
  pos = ftell (fid);

  ## Actual textscan call including "literals" in the format string
  C = textscan (fid, "%d / %d: %s");
  ## When experimenting with textscan just do 'fseek (fid, pos)' before each
call

  fclose (fid)

  ## Rework last column (read as text) into doubles and concatenate
textscan's otput
  C = [ C{1} C{2} (cell2mat (cellfun (@str2double, C{3}, "uni", 0))) ];


Philip



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html