Problem using fscanf

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem using fscanf

Utherr13
I have a file which has integers in a 100x2 matrix. I want to get the second column and put it in a variable, in which i used a for 1:100, but on the i+1 (from the second) the variable is "[](0x1)". The first numbers (first row) is ok.

for i=1:100
                z = fscanf(f,'%d',1);
                a = fscanf(f,"%d",1);
                disp(a);
endfor
Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Doug Stewart-4


On Thu, Apr 14, 2011 at 6:34 AM, Utherr13 <[hidden email]> wrote:
I have a file which has integers in a 100x2 matrix. I want to get the second
column and put it in a variable, in which i used a for 1:100, but on the i+1
(from the second) the variable is "[](0x1)"

for i=1:100
               z = fscanf(f,'%d',1);
               a = fscanf(f,"%d",1);
               disp(a);
endfor

--

Try This

for i=1:100
               z(i) = fscanf(f,'%d',1);
               a(i) = fscanf(f,"%d",1);
               disp(a);
endfor
 

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Bård Skaflestad
On Thu, 2011-04-14 at 15:11 +0200, Doug Stewart wrote:

>
>
> On Thu, Apr 14, 2011 at 6:34 AM, Utherr13 <[hidden email]>
> wrote:
>         I have a file which has integers in a 100x2 matrix. I want to
>         get the second
>         column and put it in a variable, in which i used a for 1:100,
>         but on the i+1
>         (from the second) the variable is "[](0x1)"
>        
>         for i=1:100
>                        z = fscanf(f,'%d',1);
>                        a = fscanf(f,"%d",1);
>                        disp(a);
>         endfor
>        
>         --
>
>
> Try This
>
>
> for i=1:100
>                z(i) = fscanf(f,'%d',1);
>                a(i) = fscanf(f,"%d",1);
>                disp(a);
> endfor
>  

Or maybe even this (the [2,100] shape + transpose is essential):

   t = fscanf(f, '%d', [2, 100]) .';
   z = t(:,1);
   a = t(:,2);  disp(a)


Sincerely,
--
Bård Skaflestad <[hidden email]>
SINTEF ICT, Applied Mathematics

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Utherr13
It only shows me the first line. I want all lines from the file.

The file looks like this:

1700   12.0\n
1701    1.0\n
1702  129.0\n

Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Bård Skaflestad
On Fri, 2011-04-15 at 07:34 +0200, Utherr13 wrote:
> It only shows me the first line. I want all lines from the file.

What is "it"?  Your original code?

Did you try my suggestion?  Here's a more complete demonstration.  Maybe
you can pinpoint the spot where this approach does not produce the
results you expected:

   # Create test data file:
   x = floor(100 * rand([100, 2]));  # create some data
   f = fopen("data.txt", "w");       # error checking omitted
   fprintf(f, "%d %d\n", x .');      # output data to file
   fclose(f);

   # Read data back in:
   f = fopen("data.txt", "r");       # error checking omitted
   t = fscanf(f, "%d", [2, 100]) .'; # [2,100]+transpose essential
   fclose(f);
   
   # Compare to original
   all(all(t == x))

By the way, this problem might be easier to attack using the "load"
function.


Sincerely,
--
Bård Skaflestad <[hidden email]>
SINTEF ICT, Applied Mathematics

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

octave from java

frank buloup
Hello,

I'm trying to use Octave from java but I'm facing a strange problem that
have already
been encountered. From java program, I'm using Runtime exec to start an
external
process, getting back input, output and error streams; sending command
using
output stream and parsing the result from input stream. Everything is
working fine
until a bad command is sent to Octave (a command containing a call to a
function
from a package that has not been installed for instance) : octave seems
to crash
and I 've not found any log file that could help me to debug something.

This problem has already been encountered, please see thread :

http://octave.1599824.n4.nabble.com/Octave-from-Java-td1630041.html#a1630055

Two points are mentioned from this thread :

1) Something has already been done in python using pipes or sockets - At
the begining of the thread
2) Using Runtime exec approach, you must use "--interactive" option and
read data in bytes - At the end of the thread

My questions :

1) Is anybody know how to use pipes or sockets and can give me some
links, starting points
that could help me ?

2) I did not understood the read "data in bytes". There is no
ByteBufferedReader class and
from a BufferedReader there exist only a read(char[],offset,length)
method, but it reads chars,
not bytes !!! Must we use a BufferedInputStream instead ?

Thanks a lot for your help,
frank
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: octave from java

Georgios Kousiouris

Hi,

are you sure you are capturing output and error streams efficiently?
There are many ways to do that, the most stable (we have tested it for
hours of execution of octave through Java) is raising threads in order
to capture output and error streams. You can find more info in the
following link:
  http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=4

You can also use java to octave library
(http://kenai.com/projects/javaoctave/downloads), for parsing some info
from java to octave.

We have also made a stable service oriented implementation of octave
execution through web services and java. You can find an initial version
of the code here:
http://irmosserviceeng.svn.sourceforge.net/viewvc/irmosserviceeng/trunk/Mapping/src/

that illustrates the usage of threads in order to raise octave
executions and capture efficiently output and error streams.

hope it helps,
George


On 4/15/2011 11:47 AM, frank buloup wrote:

> Hello,
>
> I'm trying to use Octave from java but I'm facing a strange problem
> that have already
> been encountered. From java program, I'm using Runtime exec to start
> an external
> process, getting back input, output and error streams; sending command
> using
> output stream and parsing the result from input stream. Everything is
> working fine
> until a bad command is sent to Octave (a command containing a call to
> a function
> from a package that has not been installed for instance) : octave
> seems to crash
> and I 've not found any log file that could help me to debug something.
>
> This problem has already been encountered, please see thread :
>
> http://octave.1599824.n4.nabble.com/Octave-from-Java-td1630041.html#a1630055 
>
>
> Two points are mentioned from this thread :
>
> 1) Something has already been done in python using pipes or sockets -
> At the begining of the thread
> 2) Using Runtime exec approach, you must use "--interactive" option
> and read data in bytes - At the end of the thread
>
> My questions :
>
> 1) Is anybody know how to use pipes or sockets and can give me some
> links, starting points
> that could help me ?
>
> 2) I did not understood the read "data in bytes". There is no
> ByteBufferedReader class and
> from a BufferedReader there exist only a read(char[],offset,length)
> method, but it reads chars,
> not bytes !!! Must we use a BufferedInputStream instead ?
>
> Thanks a lot for your help,
> frank
> _______________________________________________
> Help-octave mailing list
> [hidden email]
> https://mailman.cae.wisc.edu/listinfo/help-octave
>
>


--

---------------------------

George Kousiouris
Electrical and Computer Engineer
Division of Communications,
Electronics and Information Engineering
School of Electrical and Computer Engineering
Tel: +30 210 772 2546
Mobile: +30 6939354121
Fax: +30 210 772 2569
Email: [hidden email]

National Technical University of Athens
9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Utherr13
In reply to this post by Bård Skaflestad
I don't know how to use  load, help manual is ambiguous and not helping. I want some examples.

 # Create test data file:
   x = floor(100 * rand([100, 2]));  # create some data
   f = fopen("data.txt", "w");       # error checking omitted
   fprintf(f, "%d %d\n", x .');      # output data to file
   fclose(f);

whooo... hold on.. my data is not like that. It has 2,3 or 4 spaces between first column and second.

The piece of code that you supplied to me only gets the first row.

Here's my complete code that doesn't work:

function y=dens_energy()
        % a = 3
        % besseli(0,3) = 4,8808
        f=fopen('sunspot.dat','r');
        a=zeros(100,2);
        for i=1:100
                z = fscanf(f,"%d",1); %first column is useless
                a(i) = fscanf(f,"%d",1);
                disp(a(i));
        endfor
endfunction

Error generated : error: A(I) = X: X must have the same size as I (line 8)

a part of sunspot.dat:

1701   11.0
1702   16.0
1703   23.0
1704   36.0
1705   58.0
1706   29.0
1707   20.0
1708   10.0
1709    8.0
1710    3.0
1711    0.0
1712    0.0
1713    2.0
1714   11.0
Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Bård Skaflestad
On Fri, 2011-04-15 at 11:37 +0200, Utherr13 wrote:

> I don't know how to use  load, help manual is ambiguous and not helping. I
> want some examples.
>
>  # Create test data file:
>    x = floor(100 * rand([100, 2]));  # create some data
>    f = fopen("data.txt", "w");       # error checking omitted
>    fprintf(f, "%d %d\n", x .');      # output data to file
>    fclose(f);
>
> whooo... hold one.. my data is not like that. It has 2,3 or 4 spaces between
> first column and second.

For the purpose of "fscanf" your data is *exactly* like that.  A single
space in the scan template is interpreted as an arbitrary (positive)
number of space-like characters.  See for instance

http://www.gnu.org/software/octave/doc/interpreter/Formatted-Input.html

or any text book on C.

>
> The piece of code that you supplied to me only gets the first row.
>
> Here's my complete code that doesn't work:
>
> function y=dens_energy()
> % a = 3
> % besseli(0,3) = 4,8808
> f=fopen('sunspot.dat','r');
> a=zeros(100,2);
> for i=1:100
> z = fscanf(f,"%d",1); %first column is useless
> a(i) = fscanf(f,"%d",1);
> disp(a(i));
> endfor
> endfunction

Right.  So you're (attempting) to read lines of integers (%d) from the
file sunspot.dat

>
> a part of sunspot.dat:
>
> 1701   11.0
> 1702   16.0

except sunspot.dat does not contain (only) integers.  The result is
input failure.  You need to use a scan template for floating point
conversion to read this data.  Try this:

   t = fscanf(f, "%f", [2, inf]) .';
   a = t(:,2);

The "%f" template converts integer data too.

Still, using "load" is probably easier for this task as you don't have
to remember to close the file once you're done with it:

   a = load("sunspot.dat");
   a = a(:,2);


Sincerely,
--
Bård Skaflestad <[hidden email]>
SINTEF ICT, Applied Mathematics

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: octave from java

Kim Hansen-3
In reply to this post by frank buloup
On Fri, Apr 15, 2011 at 10:47, frank buloup <[hidden email]> wrote:

> Hello,
>
> I'm trying to use Octave from java but I'm facing a strange problem that
> have already
> been encountered. From java program, I'm using Runtime exec to start an
> external
> process, getting back input, output and error streams; sending command using
> output stream and parsing the result from input stream. Everything is
> working fine
> until a bad command is sent to Octave (a command containing a call to a
> function
> from a package that has not been installed for instance) : octave seems to
> crash
> and I 've not found any log file that could help me to debug something.
>
> This problem has already been encountered, please see thread :
>
> http://octave.1599824.n4.nabble.com/Octave-from-Java-td1630041.html#a1630055

The problem with handling bad commands has been solved in JavaOctave
(http://kenai.com/projects/javaoctave/). Have you tried JavaOctave?

Regards,
--
Kim Hansen
Vadgårdsvej 3, 2.tv
2860 Søborg
Phone: +45 3091 2437
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: octave from java

Utherr13
THanks, I solved my problems. I need to use octave only (for college).
Reply | Threaded
Open this post in threaded view
|

Re: octave from java

frank buloup
In reply to this post by Kim Hansen-3
Le 15/04/11 14:08, Kim Hansen a écrit :
On Fri, Apr 15, 2011 at 10:47, frank buloup [hidden email] wrote:
Hello,

I'm trying to use Octave from java but I'm facing a strange problem that
have already
been encountered. From java program, I'm using Runtime exec to start an
external
process, getting back input, output and error streams; sending command using
output stream and parsing the result from input stream. Everything is
working fine
until a bad command is sent to Octave (a command containing a call to a
function
from a package that has not been installed for instance) : octave seems to
crash
and I 've not found any log file that could help me to debug something.

This problem has already been encountered, please see thread :

http://octave.1599824.n4.nabble.com/Octave-from-Java-td1630041.html#a1630055
The problem with handling bad commands has been solved in JavaOctave
(http://kenai.com/projects/javaoctave/). Have you tried JavaOctave?

Regards,
Hello Kim,

thanks a lot for your reply, you gave me the solution : wrap every
sent commands in an "eval()" function. THANKS YOU VERY MUCH !!!!!!!

I was aware of the existence of javaoctave, but never tried it. The solution
that I've used is not so generic as your and is very linked to our needs.

But when I was looking for the solution, I've downloaded src of javaoctave and
had a look at it. I've seen that you've wrapped almost every octave type in a specific
octave class. Why did you choose to do that ? Was it impossible to wrap octave double, or
matrix of doubles in native java type double and double[] for instance ? Have you ever test your
code in manipulating a huge amount of data (say 500*60000 array of double for instance) ?

Again thanks a lot for your help,
With best regards,
frank

--

Frank Buloup

04 91 17 22 71

Institut des Sciences du Mouvement

UMR 6233 CNRS & Université de le Méditerranée

163 avenue de Luminy

13288 Marseille Cedex 9

Fax : 04 91 17 22 52


_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Jordi Gutiérrez Hermoso-2
In reply to this post by Utherr13
On 15 April 2011 04:37, Utherr13 <[hidden email]> wrote:
> I don't know how to use  load, help manual is ambiguous and not helping.

Can you please explain what part of the manual is ambiguous so we can
improve it?

> I want some examples.

That's fair, the entire manual could use many more examples.

- Jordi G. H.
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Problem using fscanf

Bård Skaflestad
In reply to this post by Bård Skaflestad
On Fri, 2011-04-15 at 12:23 +0200, Bård Skaflestad wrote:
> Still, using "load" is probably easier for this task as you don't have
> to remember to close the file once you're done with it:
>
>    a = load("sunspot.dat");
>    a = a(:,2);

On the other hand, using "fscanf" allows you to do this if the first
column is truly inconsequential in your application

   a = fscanf(f, "%*d %f");

This may be significant (in terms of memory) if your input ever grows
large (several million rows).


Sincerely,
--
Bård Skaflestad <[hidden email]>
SINTEF ICT, Applied Mathematics

_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: octave from java

Kim Hansen-3
In reply to this post by frank buloup
On Fri, Apr 15, 2011 at 15:18, frank buloup <[hidden email]> wrote:
>
> thanks a lot for your reply, you gave me the solution : wrap every
> sent commands in an "eval()" function. THANKS YOU VERY MUCH !!!!!!!

Yes, that trick solved a lot of problems for us, it might cost a
little in performance but I don't know if it is more or less than the
cost of using pipes to talk to octave.

> I was aware of the existence of javaoctave, but never tried it. The solution
> that I've used is not so generic as your and is very linked to our needs.
>
> But when I was looking for the solution, I've downloaded src of javaoctave
> and
> had a look at it. I've seen that you've wrapped almost every octave type in
> a specific
> octave class. Why did you choose to do that ? Was it impossible to wrap
> octave double, or
> matrix of doubles in native java type double and double[] for instance ?

We have a lot of good reasons for doing it, and not really a reason
for not doing it as the wrappers are very cheap.
It makes it a lot easier to use the structs that we have a common
ancestor. Out matrix class uses 1-base indexes in order to look more
like octave and it is resizeable, we could not get that from
double[][].

> Have you ever test your
> code in manipulating a huge amount of data (say 500*60000 array of double
> for instance) ?

I just tested it on a small program, it took 50 seconds to put() data
from Java to Octave, and 175 seconds for get() on a fairly slow
laptop. It looks like the limiting factor is octave, it uses 100% cpu
during the entire transfer. If I load or save the same data to a file
in the -text format it takes 25 and 50 seconds, it might be possible
to get the transfer times down to that by looking at the buffering
between Java and octave. Using the same data and loading from stdin
and saving to stdout takes 45 and 150 seconds, I don't really know how
to interpret that, perhaps there should be added some extra buffering
in octave when saving to stdout.

--
Kim Hansen
Vadgårdsvej 3, 2.tv
2860 Søborg
Phone: +45 3091 2437
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave