Octave/Matlab gcc front end?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Octave/Matlab gcc front end?

niles-3

I've seen many engineers spending lots of time converting large
Matlab/Octave m-files into C or Ada code to make them run faster.

As just as an idea, would it be possable to make an Octave front end
for gcc so you could compile your matlab functions?  Would it
improve preformance significantly?  I would think so if there are many
for loops.  In a way this is a variation of the MEX-file concept, but
in this way your m-file could be independent programs.

What do you think?

        Thanks,
        Rick Niles.


Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

Ted.Harding
> I've seen many engineers spending lots of time converting large
> Matlab/Octave m-files into C or Ada code to make them run faster.
>
> As just as an idea, would it be possable to make an Octave front end
> for gcc so you could compile your matlab functions?  Would it
> improve preformance significantly?  I would think so if there are many
> for loops.  In a way this is a variation of the MEX-file concept, but
> in this way your m-file could be independent programs.
>
> What do you think?
>
> Thanks,
> Rick Niles.

It would improve performance enormously in some cases. I have encountered
cases where the m-file takes many minutes (even up to an hour), and it is
quicker to put the data out to a file, pass this file to a C program which
does the computation (use the "system" command) and passes it back through
a file which octave re-loads (a matter of seconds). This in fact is one
way of doing it (you just make sure the C program writes the file in the
right format), but it clashes a bit with the philosophy of a program like
octave.

It would be a great help if this were better integrated into octave.

Ted.                                    ([hidden email])

Reply | Threaded
Open this post in threaded view
|

Octave/Matlab gcc front end?

Alexey Goldin
In reply to this post by niles-3
[hidden email] writes:
 >
 > I've seen many engineers spending lots of time converting large
 > Matlab/Octave m-files into C or Ada code to make them run faster.
 >
 > As just as an idea, would it be possable to make an Octave front end
 > for gcc so you could compile your matlab functions?  Would it
 > improve preformance significantly?  I would think so if there are many
 > for loops.  In a way this is a variation of the MEX-file concept, but
 > in this way your m-file could be independent programs.
 >
 > What do you think?
 >
 > Thanks,
 > Rick Niles.
 >

There exist compiler Matlab->C++, called Matcom. Unfortunately
it works on IBM PC with DOS or Windows ( all the same for me --
I am using Linux). May be easier to make Octave->C (C++)
compiler then front end for GCC.

Actually I am not sure it would be ALWAYS faster. I am running
some code where botttleneck is FFT transform. When I converted
it into C, it ran 1.5 times slower!!!! Looks like Octave binary
that I was using had libcruft compiled with Fortran, and C
program was using the same FFTPACK that Octave used but
converted with f2c.  (BTW -- in file PROJECTS in Octave
distribution JWE suggested that it would be a great idea to get
rid from Fortran files and use C instead. NO, no and no. If
someone cares about pure folks who do not have fortran compiler
---  there is f2c and GNU Fortran, beta version, but quite
usable).


But, the C program was using 5 times less memory.


Of course if there are a lot of loops  or calls to functions
like quad, fsolve, etc. C code will win a lot.





Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

Alexey Goldin
In reply to this post by Ted.Harding
Ted Harding writes:
 >
 > It would improve performance enormously in some cases. I have encountered
 > cases where the m-file takes many minutes (even up to an hour), and it is
 > quicker to put the data out to a file, pass this file to a C program which
 > does the computation (use the "system" command) and passes it back through
 > a file which octave re-loads (a matter of seconds). This in fact is one
 > way of doing it (you just make sure the C program writes the file in the
 > right format), but it clashes a bit with the philosophy of a program like
 > octave.
 >
 > It would be a great help if this were better integrated into octave.
 >
 > Ted.                                    ([hidden email])

Much better to use dld. Unfortunately it does not work on all systems.

Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

Ted.Harding
( Re Message From: Alexey Goldin )
> Ted Harding writes:
>  >
>  > It would be a great help if this were better integrated into octave.
>  >
>  > Ted.                                    ([hidden email])
>
> Much better to use dld. Unfortunately it does not work on all systems.
>

How right you are ... <sigh>.

Ted.                                    ([hidden email])

Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

Will Powell (CASE Research student BAe)
In reply to this post by niles-3
> I've seen many engineers spending lots of time converting large
> Matlab/Octave m-files into C or Ada code to make them run faster.

> As just as an idea, would it be possable to make an Octave front end
> for gcc so you could compile your matlab functions?  Would it
> improve preformance significantly?  I would think so if there are many
> for loops.  In a way this is a variation of the MEX-file concept, but
> in this way your m-file could be independent programs.

At the moment I'm still a "mostly Matlab" user, rather than Octave user.
I've been using Matlab and cmex with the Fortran NAG Libraries to do
some nasty integrations for some statistical modelling work. This is
all fine at University where we have Matlab & NAG site-lisenced, but
I've been looking for a replacement for this set-up at home. Sometimes
I do use Octave for simple jobs on my Linux box at home, but it's
always seemed a bit of a shame not to have some lower-level way of
writing programs to be callable from octave's command line.

I do like the MEX philosophy, but there again, maybe I've been using matlab
just too long...

Will.

Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

John W. Eaton-6
In reply to this post by niles-3
First, I would like to improve Octave's ability to load externally
compiled functions.  If dld is never going to be ported to more
systems, it would probably be good to use dlopen, or other
system-specific utilities.  I think that would allow dynamic linking
on most systems of interest.  Is there anyone out there who has
experience using these tools who would be willing to contribute code,
or adapt existing solutions from other systems like perl or gnans?

I would also like to make it possible to use Octave to generate C++
code directly.  The idea is that the generated code could be
dynamically linked or used as stand alone programs.  Whether this will
be faster than using Octave directly depends on a lot of things.  For
loops would be a lot faster if the translator can determine that it
can use integers for the iteration variable, but will probably not be
much faster for things like

  a = rand (10,100);
  for i = a
    # do something with each column of a in turn.
  end

because all of the work to extract the columns of the matrix have to
be done anyway.  (Of course, Octave can probably be made to work a bit
faster than it currently does for this case too...)  This project is
likely to involve quite a bit of work.

Also, about translating to various parts of Octave to C, the PROJECTS
file entry says

  * Translate Fortran routines to C/C++, or replace them entirely.

I've never had plans to use f2c for this.  My intent was only to
replace the code if it would make it easier to maintain.  Until that
is possible, I'm happy to stick with the Fortran source.

jwe

Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

John Utz-3
In reply to this post by niles-3
Hi Niles;

A darn good idea, but probably much easier said then done :-(


On Tue, 12 Sep 1995 [hidden email] wrote:

>
> I've seen many engineers spending lots of time converting large
> Matlab/Octave m-files into C or Ada code to make them run faster.

        I myself have actually thought of this as being the natural
course of events.. octave, matlab/tutsim.. et all provide one with a
chance to beat on ideas and try and quickly evaluate ( relatively
speaking :-) ) their possiblity of success by evaluating the numerical
and graphical results that can be easily ( again, relativly speaking :-) )
generated by these tools.

> As just as an idea, would it be possable to make an Octave front end
> for gcc so you could compile your matlab functions?

        I think this might be pretty complex. I have watched the mail
concerning the development of the *fortran 77* ( g77 ) frontend to gcc
and have seen the suprising degree of difficulty in getting even a
relativly old ( 35 years, remember? ) and primitive language that was
designed * at the outset * to have a near correspondence to native (
assembly ) instructions to work with gcc.

        To take a highly abstracted language such as octave and build a
front end would be much more difficult ( in my humble and largely
ignorant of the specific issues opinion ).

> Would it improve preformance significantly?  I would think so if there are
> many for loops.

        Oh, heck yes! Remember, matlab and octave are * interpreters *.
not compilers. This means that for each line the interpreter must read,
parse and execute as opposed to reading the entire file, parsing it once,
and spending a substantial period of time converting the hi level
instructions into native format that, once converted, are available, in
effect, forever in their predigested form.

        This also means that since octave has no " overall picture "
 of where the function is going ( since it is looking only at one line
at a time ) it is unable to do all the nifty optimizations ( in
peoplespeak : shortcuts! ) that a compiler can make since it knows the
beginning and end of the journey, so to speak.

        Your point about loops is also well taken for the reasons stated
in the above paragraphs, only ever so much more so!

        octave and matlab are like gwbasic or qbasic on an msdos machine,
as opposed to quickC or QuickBasic on the same platform.

        This is why matlab has a fortran hook, and jwe has c++ hooks for
octave planned or implemented, i forget which :-) .

> In a way this is a variation of the MEX-file concept, but
> in this way your m-file could be independent programs.
> What do you think?
>
> Thanks,
> Rick Niles.
>
>

*******************************************************************************
 John Utz [hidden email]
        idiocy is the impulse function in the convolution of life


Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

Ted.Harding
In reply to this post by niles-3
( Re Message From: John Eaton )

> First, I would like to improve Octave's ability to load externally
> compiled functions.  If dld is never going to be ported to more
> systems, it would probably be good to use dlopen, or other
> system-specific utilities.  I think that would allow dynamic linking
> on most systems of interest.  Is there anyone out there who has
> experience using these tools who would be willing to contribute code,
> or adapt existing solutions from other systems like perl or gnans?

May I ask a naive question? In Matlab, the MEX file mechanism, though
needing some care, works very straightforwardly. Knowing the structures
and parameter-passing method used by Matlab, you write (in FORTRAN or C) a
program to do what you want and pass back the results. You then compile
this into object code, using a couple of Matlab-supplied libraries.  This
works very smoothly even in DOS: the batch file (from 1988!) for Turbo C
is simply

C:\TC\TCC -a -c -f87 -ml -N- -IC:\TC\INCLUDE %1
IF ERRORLEVEL 1 GOTO DONE
C:\TC\TLINK C:\ML\MEX\CMEX+C:\TC\LIB\C0L+%1,%1.MEX @C:\ML\MEX\TCMEX.RSP /c /m;
C:\ML\MEX\PROT_MEX %1
DEL %1.OBJ
:DONE

(The program protmex.exe does something which I've forgotten, but it's
supplied.)

Matlab seems to have a builtin method of accessing the resulting .MEX object
file (over-lay style, I suppose). Although this is not quite the same as
"installing a new built-in function on the fly", it works very well.

Now my naive question is: If it is this simple in DOS/Matlab, why does it
seem to be so problematic with UNIX/octave, with fuss about dld etc?

Please note I'm not suggesting there is no problem: but I'm interested to
know. Or are we being more ambitious than Matlab+MEX?

Ted.                                    ([hidden email])

Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

Frederic Devernay-3
a simple solution would be to make MEX files executables that would
communicate with matlab through stdin/stdout, this way there would be
no dld problems at all and it would be a lot more portable. Maybe we
could even have the same API as mex files in Matlab so that extensions
could be compiled either for matlab or octave.

fred
--
       *       mailto:[hidden email]      *
       *   http://www.inria.fr/robotvis/personnel/devernay/  *
       * Projet ROBOTVIS -- INRIA Sophia Antipolis -- FRANCE *

Reply | Threaded
Open this post in threaded view
|

Re: Octave/Matlab gcc front end?

John W. Eaton-6
In reply to this post by Ted.Harding
Ted Harding <[hidden email]> wrote:

: May I ask a naive question? In Matlab, the MEX file mechanism, though
: needing some care, works very straightforwardly.

It is currently possible (though not well documented or particularly
well tested) to do essentially the same thing with Octave, provided
that you have dld available.  To make this work for other systems that
don't have dld but do have some other method of dynamic linking should
not be too hard, but like everything else, it takes time and there are
lots of other things in the PROJECTS file and in my list of pending
bug reports and other messages...

: Now my naive question is: If it is this simple in DOS/Matlab, why does it
: seem to be so problematic with UNIX/octave, with fuss about dld etc?

I don't see that there is much difference.  What you don't see in the
DOS/Matlab code is what Matlab actually has to do to link to the .mex
file.  With Octave, you can see what is required.  The code is in the
file src/dynamic-ld.cc (the glue between Octave and any dynamic
linking subroutines) and in the dld subdirectory (the code for all the
actual dirty work).

The dynamic-ld.cc file currently has the following code:

  Octave_builtin_fcn
  load_octave_builtin (const char *name)
  {
  #ifdef WITH_DLD
    return dld_octave_builtin (name);
  #else
    return 0;
  #endif
  }

  int
  load_octave_oct_file (const char *name)
  {
  #ifdef WITH_DLD
    return dld_octave_oct_file (name);
  #endif
    return 0;
  }

  void
  init_dynamic_linker (void)
  {
  #ifdef WITH_DLD
    octave_dld_init ();
  #endif
  }

along with the definitions of octave_dld_init(), dld_octave_oct_file(),
and dld_octave_builtin().

It should be possible to make dynamic linking work with some mechanism
other than dld simply by adding new cases to the functions shown
above, implementing the necessary functions corresponding to the ones
for dld, and possibly changing the configuration to build the right
kinds of objects/libraries for dynamic linking of the `builtin'
functions.  It would probably even be possible to support more than
one kind of dynamic linking on the same platform if that would be
useful.

jwe