Octave Compiler

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Octave Compiler

Jens Ruecknagel-2
To whom in concerns,

As a student research project I want to develop an Octave to C++ compiler.

About me:
My name is Jens Ruecknagel. I am a student at Technical University of
Ilmenau, Germany in the course computer science in my 4th and final year.
After finishing my degree I am going to be am diploma computer scientist
(comparable to a master degree). I already have some experience in
developing programming languages.

My project:
I am trying to adapt the Lex/Yacc grammar of Octave to fit the needs of an
Octave to C++ compiler. And I am developing a backend to generate
C++ code.

I am just starting to work on this project, therefore I am still “reading”
the Octave code to find out which parts of the Octave project I can use.

I think, I can use the Lex/Yacc parser which (I suppose) constructs an
abstract syntax tree which then will be feed to the my compilers backend.
I’m still thinking about which numerical libs the generated code shall
depend on – I’ll probably use liboctave first and maybe replace it later.


So I hope you answer some questions about Octave:

Do you have any additional documentation on the Octave source code?
On which Octave version should the compiler be based on? I was thinking I
use 2.1.50.
Can you give me some hints about my task?



May I bother you again, if I run into problems? (I am sure there will be
some problems :-))


Tanks a lot,

Jens Ruecknagel


Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

JD Cole
Jens,
    I've looked (and done) a little bit of work in this same direction.
Are you talking about writing an actual compiler, or an Octave to C++
code generator? In the case of the latter, you might consider
integrating the code generator into the main Octave source. That said,
when developing a new, semi-large addition to this source tree,
recompilation is a bit cumbersome, so, what I have taken to doing is
developing via ".oct" files. This should give you access to all of the
Octave internals,  while allowing you to only recompile your source
code. (Take note that there are a few gotcha's in developing this way, I
recall having some trouble when I wanted to check if a symbol was a
function or not.)
    Since you're probably going to start with parsing, you should
familiarize yourself with the parse-tree structures, base class is
"tree" and tree walking stuff, base class "tree_walker", found in "pt.h"
and "pt-walk.h", respectively. The tree_walker class allows you to visit
each node in the parse tree in an ordered fashion. I will attach the bit
of code which I started. The code is basically what you find in
pt-pr-code.cc, with a few additions so I can see what goes where. You
should be able to just do a 'make' to produce 'compile.oct" and then,
using the example 'mysum.m', start up octave and 'compile('mysum')', it
should produce just a simple template and then some debug information.
(The code worked as of Octave version 2.1.45, but I haven't tried it
with new versions yet.) If you're going to compile it, you'll also want
to remove any references to "parallel" in pt-emit-cplusplus, these are
additions I've made to try to give Octave some parallel support.
    I'd be happy to work with you on this project, if you're so
inclined, or, at the very least, pass on the other tidbits I'd been
thinking about with respect to octave to c++ converter. That will just
take a little revival.
    There are two quite interesting alternatives to converting to C++
that I had been kicking around. One was creating a Just-In-Time compiler
for Octave. The other is actually writing a front end for gcc to compile
directly to binary from octave code. While the former could potentially
provide a good performance increase for the standard Octave user, the
latter may not be relevant, it just sounds like a cool project to me.
There is plenty of literature on the JIT compiler in regards to Matlab.

Hope this helps,
JD
Jens Ruecknagel wrote:

> To whom in concerns,
>
> As a student research project I want to develop an Octave to C++
> compiler.
>
> About me:
> My name is Jens Ruecknagel. I am a student at Technical University of
> Ilmenau, Germany in the course computer science in my 4th and final
> year. After finishing my degree I am going to be am diploma computer
> scientist (comparable to a master degree). I already have some
> experience in developing programming languages.
>
> My project:
> I am trying to adapt the Lex/Yacc grammar of Octave to fit the needs
> of an Octave to C++ compiler. And I am developing a backend to
> generate C++ code.
>
> I am just starting to work on this project, therefore I am still
> “reading” the Octave code to find out which parts of the Octave
> project I can use.
>
> I think, I can use the Lex/Yacc parser which (I suppose) constructs an
> abstract syntax tree which then will be feed to the my compilers backend.
> I’m still thinking about which numerical libs the generated code shall
> depend on – I’ll probably use liboctave first and maybe replace it later.
>
>
> So I hope you answer some questions about Octave:
>
> Do you have any additional documentation on the Octave source code?
> On which Octave version should the compiler be based on? I was
> thinking I use 2.1.50.
> Can you give me some hints about my task?
>
>
>
> May I bother you again, if I run into problems? (I am sure there will
> be some problems :-))
>
>
> Tanks a lot,
>
> Jens Ruecknagel
>
>


oct-compiler.tar.gz (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

JD Cole
Hi back Jens,

> thanks for your mail. Thanks for the oct-compiler frame. This really
> gave me a clue, about the architecture of octave.
> I finally got it running and now I am finding out, what in detail you
> were doing.

Well, the truth is, I hadn't done so much on that code, for the most
part I too was trying to get a good feel for how to attack that probem.

But first I'll explain some more about my project:

JD Cole wrote:

>> There are two quite interesting alternatives to converting to C++
>> that I had been kicking around. One was creating a Just-In-Time
>> compiler for Octave. The other is actually writing a front end for
>> gcc to compile directly to binary from octave code. While the former
>> could potentially provide a good performance increase for the
>> standard Octave user, the latter may not be relevant, it just sounds
>> like a cool project to me.
>
>
> The latter is much more than just a cool project - this way octave can
> make its way out of the laboratory. If you can generate C++ from Octave
> code you can take it and plug it seamless into your software project.

I agree with you here, and it was definitely one of the reasons  octave
-> C++ would be cool. As an intermediate step, it may not be "too"
difficult to convert directly to Octave-based stand alone functions,
however, since we'd be linking with the Octave libraries, the generated
code would still fall under GPL since the whole of Octave source is GPL.
(As opposed to LGPL. If you're not clear on this I'd be happy to explain
more.) A similar problem would arise when you tried to convert a script
file which makes use of other scripts which also fall under
GPL...especially so if those were .oct files which could also be
converted using the compiler.

Here's a big question: If I write a script file and put it under GPL and
you convert it to pure C++ code, does that C-code fall under GPL. My
guess is yes, since you are in effect duplicating the algorithm which I
developed. Well, maybe....it deserves a little more thought.

    That said, I think that the response from the user community would
be huge just to allow anyone to compile their script file into a .oct
file and benefit from the speedup. I had a convesation with someone and
they said a simple "no optimization" C++ implementation of some of their
.oct files gave something like a 6x speedup. This actually sounds like a
good place to start...create a .m ---> .cc ---> .oct "compiler". It
would give the user community a pretty immediate benefit and would give
the compiler authors a good stepping stone to understanding how to
convert to independent C++.

    [My interest] While I would love to reap the benefits of converting
Octave code to C++, I also have another aim, as I told you I've put some
time into adding parallel support to Octave. My grand view is to allow
Octave users to make use of large parallel systems to do their
computations. However, large simulations take a lot of time and money
(if it's a lease type situation), plus it isn't always possible to
install a large application such as Octave for single run. So I'm
thinking it may be possible to pre-compile the code which will run on
remote nodes into a form independent of octave and then execute it
during an Octave session. (This idea is still a little gray....in the
short term I'm just working on getting a bunch of machines running
Octave to cooperatively run arbitrary code.)

>
> Today developers don't use Octave because they have to reingineer their
> prototype sooner or later in C++. So they take C++ for the prototype as
> well.

Yes, I know that one well, I found myself procrastinating in converting
my working Octave code to C/C++ code for an application a few days ago.
It just hurts.

>
> Do you know Ptolemy? Ptolemy is a simulation environment. Its kind alike
> Simulink - but much more powerful. Its based on C++ classes connected by
> Tcl Functions. But C++ has limitation in numerics - its just to far away
> from the problems of engineers. So we want to use Octave as the language
> for describing the blocks. Just like Simulink is able to use Matlab.

Ya, I have come across this project before, it's a moutain they've done.
Definitely a good direction to go in, however, it may again be worth
investigating various license compatibilies....'cause the purists will
get you later on. I'm basically saying that  UC Regents copyrights and
GPL may not mix...but I'm no lawyer. (Historically UC Regents, i.e. the
governing body of all the Univesities of California has had a pretty
tight fist on anything coming out of the university. This copyright
seems to be a little more lenient, though.) Sorry, I've been reading so
much about court cases lately all this stuff is coming out my ears!

>> There is plenty of literature on the JIT compiler in regards to Matlab.
>>
> Do you have some links?

Ya, the best one I recall is here:

http://www.cs.uiuc.edu/Dienst/UI/2.0/Describe/ncstrl.uiuc_cs/UIUCDCS-R-2001-2213

and if you're into that check out GNU Lightning:

http://www.gnu.org/software/lightning/lightning.html

it's a somewhat cross-platform library for generating and executing code
at run-time.

when I was looking at this stuff, someone had also suggested that rather
than create a new runtime compiler, code could be converted to an
existing one like Java....something everybody has access too and is
rather mature. (I'm not sure if this is appropriate or not, but
definitely worth some investigation.

> There is a problem, I am hanging in understanding your code and getting
> started with coding my self. I don`t have to much experience in coding
> with gnu tools. (Never did big projects)

> Do you know how I can debug .oct files? How I can trace into them with a
> debugger?

Ya. OK, I'm going to assume you have used gdb at least once, if you
haven't feel free to speak up and I give you more. Basically the problem
with .oct files is that they are dynamically linked, the implication is
that symbols and code will not be available via the debugger until
Octave has loaded that specific file. My way around this is to do the
following, let's assume we're debugging the .oct file oregano.cc (the
example included with Octave).

1. Start the debugger "gdb octave".
2. at the prompt  start octave using the "run" command
3. Once at the Octave command prompt you need to "coax" Octave into
loading the command, but if we're trying to debug something it might not
be desirable to actually execute the code, so I just type "help
oregano". This forces Octave to load the file into memory.
4. Now that oregano.oct has been loaded, do a "control-c". This should
drop you back to the debugger prompt where you should have access to the
symbols and code in your .oct file. At this point you would presumably
set breakpoint(s) within your code and continue down the debugging path.

NOTE: A couple of caveauts to debugging Octave code. 1) Octave source
makes have use  of inlined functions, so often when you arrive at a
breakpoints which lies within your code. When stepping through code I
have written, it hasn't been uncommon that I see none of my original
code. 2) Compiler optimizations will further obfuscate your code. For
example using "-g -O2" is sometimes confusing to step through.

> What development environment do you use? I'm using Emacs (due
> to my former small projects) - but  I'm still not convinced, its the
> best decision.

Well, I wouldn't call emacs a development environment, but I do use it
for all my coding. I like the fact that it works on almost every
platform I use and everything is accessible from a combination of keys
(dont let that scare you off). Plus it's the right price. There are more
"complete" development environments, but I just use a combination of
emacs and gdb in one of three forms (command line gdb, emacs gdb or DDD
which is a really nice graphical interface to gdb.) If you're used to a
VC++ development environment going to emacs may have a shock factor
since there are so many pictures.

If you would like to work to together on your idea, I'd be more than
happy to...perhaps we could turn something out that really gets people
going.

JD


Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

Przemek Klosowski
In reply to this post by Jens Ruecknagel-2

   file and benefit from the speedup. I had a convesation with someone and
   they said a simple "no optimization" C++ implementation of some of their
   ..oct files gave something like a 6x speedup.

I think the best approach is to have a flexible component system. I
think it is a false dichotomy to propose either 'full octave' or 'full
C++'. In particular, I doubt that every piece of the above-mentioned
application sped up 6 times---I would guess that they had some pieces
that were same (or slower!) in C++, and some pieces that run tight
computation loops that sped up, say, 100 times, giving an average of 6.

In other words, I think it is better to prototype in .m files, find
out the slow spots, and speed those up. This is such a conventional
advice that I feel silly writing it down---I really don't understand
why people would prefer to prototype in C++ unless they really
disliked the .m files while loving C++ a lot.

The current .oct system is a good start: it's fairly flexible, gives
easy access to most of Octave internals, has the beginnings of a
reasonable documentation, etc. Still, as the constant discussions
show, there are usability problems with it. In the glorious end,
I wish for a flexible component system with the following attributes:

 - flexible, rich and probably extensible data exchange mechanism, to
   pass data between modules

 - language, network and platform independence--this kind-of follows from
   the previous attribute

 - introspective, self-describing environment, with some kind of directory
   of available modules/services

 - good facilities for profiling (to find the weak/slow spots) and debugging
   (NB, John, is there a way to instrument Octave to do a high-level equivalent
   of gprof? Would that be a good idea? It might simplify development work
   like fixing the concat slowness that you did recently)

As it is today, we end up doing a lot of duplicate work: the current paradigm
requires 'capturing' functionality in each system (e.g., we have to 'port'
FFTW and VTk to octave; compile GUI widgets into Matlab, etc. etc.). I want
to be able to use facilities provided by different applications to be
easily cross-hooked between them. For instance, I would like to be able
to share an octave array with Gnumeric, for some ad-hoc editing.

In a sense, Octave's plotting (via gnuplot) is exactly the right approach:
"Don't duplicate! Freeload!". Of course, octave/gnuplot suffers from gnuplot's
one-directional approach (it will accept your X limits but won't tell you what its
own X limits are), and the data exchange using disk files is, ahem, limited.

Microsoft COM (and DCOM) has this to a degree, but it is a poor design
in some other respects. I hear that a number of people think that .NET
is a reasonable design (Miguel de Icaza and the MONO team)---I reserve
a judgement on that, and on how well it would work in the real world
with Linux/Mac/Windows components, but the general idea is right.

   thinking it may be possible to pre-compile the code which will run on
   remote nodes into a form independent of octave and then execute it
   during an Octave session. (This idea is still a little gray....in the
   short term I'm just working on getting a bunch of machines running
   Octave to cooperatively run arbitrary code.)

That's actually very close to what I am proposing!! don't write the whole
application in MPI, just the relevant pieces that benefit from it.

   > Today developers don't use Octave because they have to reingineer their
   > prototype sooner or later in C++. So they take C++ for the prototype as

   Yes, I know that one well, I found myself procrastinating in converting
   my working Octave code to C/C++ code for an application a few days ago.
   It just hurts.

Yes, it hurts. AArgh! Let's not do that :).

   > Do you know Ptolemy? Ptolemy is a simulation environment. Its kind alike
   > Simulink - but much more powerful. Its based on C++ classes connected by
   > Tcl Functions. But C++ has limitation in numerics - its just to far away
   > from the problems of engineers. So we want to use Octave as the language
   > for describing the blocks. Just like Simulink is able to use Matlab.

   Ya, I have come across this project before, it's a moutain they've done.
   Definitely a good direction to go in, however, it may again be worth

The usual problem with frameworks is this: one can start by
concentrating on the framework first, or the 'modules' first. In the
former case, it is hard to get people to use the framework, and it is
hard to get the design right---there's a tendency to overengineer or
get the features wrong. In the latter case, the framework tends to be
an afterthought, with, e.g., design limitations. Perhaps taking an
existing framework, like Ptolemy, or .NET, is a good approach???



Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

Przemek Klosowski
In reply to this post by Jens Ruecknagel-2

   Here's a big question: If I write a script file and put it under GPL and
   you convert it to pure C++ code, does that C-code fall under GPL. My
   guess is yes, since you are in effect duplicating the algorithm which I
   developed. Well, maybe....it deserves a little more thought.

Even if it is not murky legally, it definitely is morally---and I think that
it would be copyrighted in the legal sense too; the closest analogy that comes to
mind is a translation of a literary work. Note that in a translation there
is at least a human element, and here we are talking about an automatic process.


Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

John W. Eaton-6
In reply to this post by JD Cole
On 18-Feb-2004, JD Cole <[hidden email]> wrote:

| I agree with you here, and it was definitely one of the reasons  octave
| -> C++ would be cool. As an intermediate step, it may not be "too"
| difficult to convert directly to Octave-based stand alone functions,
| however, since we'd be linking with the Octave libraries, the generated
| code would still fall under GPL since the whole of Octave source is GPL.
| (As opposed to LGPL. If you're not clear on this I'd be happy to explain
| more.) A similar problem would arise when you tried to convert a script
| file which makes use of other scripts which also fall under
| GPL...especially so if those were .oct files which could also be
| converted using the compiler.

Yes, any translated Octave code or any code that depends on the Octave
libraries would be a derivative work, so it would have to be licensed
under terms that are compatible with the GPL.

| NOTE: A couple of caveauts to debugging Octave code. 1) Octave source
| makes have use  of inlined functions, so often when you arrive at a
| breakpoints which lies within your code. When stepping through code I
| have written, it hasn't been uncommon that I see none of my original
| code. 2) Compiler optimizations will further obfuscate your code. For
| example using "-g -O2" is sometimes confusing to step through.

I find it nearly impossible to use gdb to debug anything compiled with
g++ -g -O2.  So when I really must use a debugger to check out what is
happening inside Octave, I recompile at least all the parts I want to
debug with only -g.  Otherwise too many local variables are
unavailable and the correspondence between the source code and what
the debugger sees is terrible.

jwe


Reply | Threaded
Open this post in threaded view
|

profiling (was: Re: Octave Compiler)

John W. Eaton-6
In reply to this post by Przemek Klosowski
On 18-Feb-2004, Przemek Klosowski <[hidden email]> wrote:

| - good facilities for profiling (to find the weak/slow spots) and
|   debugging (NB, John, is there a way to instrument Octave to do a
|   high-level equivalent of gprof? Would that be a good idea? It
|   might simplify development work like fixing the concat slowness
|   that you did recently)

It would be possible to add some method of timing code to the
interpreter.  How fine-grained should the profile be?  Function calls
only or down to the individual statement level?  Should it only
profile the interpreted code or should it also time compiled code
(.oct files and built-in functions)?  At the level of compiled code,
you would only be able to get timings per function call unless you
actually compiled Octave for profiling and used the normal profiling
tools.  But from within Octave itself, we should be able to have some
profiling support.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: profiling (was: Re: Octave Compiler)

Przemek Klosowski

   | - good facilities for profiling (to find the weak/slow spots) and
   |   debugging (NB, John, is there a way to instrument Octave to do a
   |   high-level equivalent of gprof? Would that be a good idea? It
   |   might simplify development work like fixing the concat slowness
   |   that you did recently)

   It would be possible to add some method of timing code to the
   interpreter.  How fine-grained should the profile be?  Function calls
   only or down to the individual statement level?  Should it only
   profile the interpreted code or should it also time compiled code
   (.oct files and built-in functions)?  At the level of compiled code,
   you would only be able to get timings per function call unless you
   actually compiled Octave for profiling and used the normal profiling
   tools.

I would imagine that finer would be better---per-statement for .m files and
per-function for .oct files. What would be a nice interface to it? perhaps
a structure, where field names identify functions, and the value is a 2D array
whose i-th elements are the count and cumulative time in the i-th line

      profile.sum(1,1)=1      first line of function 'sum' executed once
      profile.sum(1,2)=.05    .05 seconds spent in first line of function 'sum'

How does that look? I have no idea how expensive would that be (e.g., would profiling
itself change the timing?);

Should profiling be a command-line option, or should it be a runtime on-off option,
like tic and toc?


Reply | Threaded
Open this post in threaded view
|

Re: profiling (was: Re: Octave Compiler)

John W. Eaton-6
On 18-Feb-2004, Przemek Klosowski <[hidden email]> wrote:

| I would imagine that finer would be better---per-statement for .m
| files and per-function for .oct files. What would be a nice
| interface to it? perhaps a structure, where field names identify
| functions, and the value is a 2D array whose i-th elements are the
| count and cumulative time in the i-th line
|
|       profile.sum(1,1)=1      first line of function 'sum' executed once
|       profile.sum(1,2)=.05    .05 seconds spent in first line of function 'sum'
|
| How does that look?

There are already profiling commands in Matlab, so it would probably
be good to be compatible unless we can come up with something that is
much better (and even then, we should not make it impossible for
someone to write a compatibible interface).

| I have no idea how expensive would that be
| (e.g., would profiling itself change the timing?);

I think the data should be stored internally in a special purpose data
structure, so that it can be relatively efficient.  Getting the
timestamps should not be too expensive, though I have not tried to
time calls to getrusage.  The time spent converting the internal
representation to an Octave structure array should not matter too
much, as normally one would stop profiling before examining the
results of a run.

| Should profiling be a command-line option, or should it be a runtime
| on-off option, like tic and toc?

The Matlab interface provides

  profile on
  profile off

etc.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

JD Cole
In reply to this post by John W. Eaton-6
John W. Eaton wrote:

>On 18-Feb-2004, JD Cole <[hidden email]> wrote:
>
>| I agree with you here, and it was definitely one of the reasons  octave
>| -> C++ would be cool. As an intermediate step, it may not be "too"
>| difficult to convert directly to Octave-based stand alone functions,
>| however, since we'd be linking with the Octave libraries, the generated
>| code would still fall under GPL since the whole of Octave source is GPL.
>| (As opposed to LGPL. If you're not clear on this I'd be happy to explain
>| more.) A similar problem would arise when you tried to convert a script
>| file which makes use of other scripts which also fall under
>| GPL...especially so if those were .oct files which could also be
>| converted using the compiler.
>
>Yes, any translated Octave code or any code that depends on the Octave
>libraries would be a derivative work, so it would have to be licensed
>under terms that are compatible with the GPL.
>  
>
I should mention that even with this "restriction" the benefit of pure
C++ or C++/Octave would still be a spectacular advance for the user
community, even if they couldn't distribute it as closed source. It also
should go with out saying that the Octave -> pure C++ will still be
greatly aided by a utility which adds stubs to converted code which the
user must later fill in depending on their target use. If it's a closed
source proprietary thing...then you have to do some more footwork and
implement your own fft, convolutions, etc. if it's just for your local
experimentation existing Octave modules can be used.

On the part of licensing how would such derivative could be referred to
in terms of licensing? For example I have two .m's, one from, say,
octave-forge the other is of my own creation. When I go to distribute my
code as a derivative work it would include a GPL for me, and of course,
the GPL for the octave-forge code? This maybe isn't so clear, sorry.

JD


Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

Jens Ruecknagel-2
In reply to this post by JD Cole
Hi again JD Cole,

after a lot of thinking I've concluded, that I want to build the
compiler as an .oct file - but it should produce stand alone code (maybe
based on liboctave) - but I have to check with my Professor, and I just
cannot get in touch with him.

But don't get into a hurry to code something - I want to build this
compiler in the next 3 Months - if you just answer my mail - this wouldt
be a great help.

So right now I am looking at your code. And now I have many Questions:

What is the file goal.cc for?
Is it just, what you wish compile to produce?

What does the operator |= in compile.cc do?
Is it bitwise OR and assign? Or is it logical OR and assign (sorry about
the C++ stuff, but obviously you thought about it)


More Questions as son as I ran into them and cannot figure it out.



Thanks a lot.

Jens


> Hi back Jens,
>
>> thanks for your mail. Thanks for the oct-compiler frame. This really
>> gave me a clue, about the architecture of octave.
>> I finally got it running and now I am finding out, what in detail you
>> were doing.
>
>
> Well, the truth is, I hadn't done so much on that code, for the most
> part I too was trying to get a good feel for how to attack that probem.
>
> But first I'll explain some more about my project:
>
> JD Cole wrote:
>
>>> There are two quite interesting alternatives to converting to C++
>>> that I had been kicking around. One was creating a Just-In-Time
>>> compiler for Octave. The other is actually writing a front end for
>>> gcc to compile directly to binary from octave code. While the former
>>> could potentially provide a good performance increase for the
>>> standard Octave user, the latter may not be relevant, it just sounds
>>> like a cool project to me.
>>
>>
>>
>> The latter is much more than just a cool project - this way octave can
>> make its way out of the laboratory. If you can generate C++ from Octave
>> code you can take it and plug it seamless into your software project.
>
>
> I agree with you here, and it was definitely one of the reasons  
> octave -> C++ would be cool. As an intermediate step, it may not be
> "too" difficult to convert directly to Octave-based stand alone
> functions, however, since we'd be linking with the Octave libraries,
> the generated code would still fall under GPL since the whole of
> Octave source is GPL. (As opposed to LGPL. If you're not clear on this
> I'd be happy to explain more.) A similar problem would arise when you
> tried to convert a script file which makes use of other scripts which
> also fall under GPL...especially so if those were .oct files which
> could also be converted using the compiler.
>
> Here's a big question: If I write a script file and put it under GPL
> and you convert it to pure C++ code, does that C-code fall under GPL.
> My guess is yes, since you are in effect duplicating the algorithm
> which I developed. Well, maybe....it deserves a little more thought.
>
>    That said, I think that the response from the user community would
> be huge just to allow anyone to compile their script file into a .oct
> file and benefit from the speedup. I had a convesation with someone
> and they said a simple "no optimization" C++ implementation of some of
> their .oct files gave something like a 6x speedup. This actually
> sounds like a good place to start...create a .m ---> .cc ---> .oct
> "compiler". It would give the user community a pretty immediate
> benefit and would give the compiler authors a good stepping stone to
> understanding how to convert to independent C++.
>
>    [My interest] While I would love to reap the benefits of converting
> Octave code to C++, I also have another aim, as I told you I've put
> some time into adding parallel support to Octave. My grand view is to
> allow Octave users to make use of large parallel systems to do their
> computations. However, large simulations take a lot of time and money
> (if it's a lease type situation), plus it isn't always possible to
> install a large application such as Octave for single run. So I'm
> thinking it may be possible to pre-compile the code which will run on
> remote nodes into a form independent of octave and then execute it
> during an Octave session. (This idea is still a little gray....in the
> short term I'm just working on getting a bunch of machines running
> Octave to cooperatively run arbitrary code.)
>
>>
>> Today developers don't use Octave because they have to reingineer their
>> prototype sooner or later in C++. So they take C++ for the prototype as
>> well.
>
>
> Yes, I know that one well, I found myself procrastinating in
> converting my working Octave code to C/C++ code for an application a
> few days ago. It just hurts.
>
>>
>> Do you know Ptolemy? Ptolemy is a simulation environment. Its kind alike
>> Simulink - but much more powerful. Its based on C++ classes connected by
>> Tcl Functions. But C++ has limitation in numerics - its just to far away
>> from the problems of engineers. So we want to use Octave as the language
>> for describing the blocks. Just like Simulink is able to use Matlab.
>
>
> Ya, I have come across this project before, it's a moutain they've
> done. Definitely a good direction to go in, however, it may again be
> worth investigating various license compatibilies....'cause the
> purists will get you later on. I'm basically saying that  UC Regents
> copyrights and GPL may not mix...but I'm no lawyer. (Historically UC
> Regents, i.e. the governing body of all the Univesities of California
> has had a pretty tight fist on anything coming out of the university.
> This copyright seems to be a little more lenient, though.) Sorry, I've
> been reading so much about court cases lately all this stuff is coming
> out my ears!
>
>>> There is plenty of literature on the JIT compiler in regards to Matlab.
>>>
>> Do you have some links?
>
>
> Ya, the best one I recall is here:
>
> http://www.cs.uiuc.edu/Dienst/UI/2.0/Describe/ncstrl.uiuc_cs/UIUCDCS-R-2001-2213 
>
>
> and if you're into that check out GNU Lightning:
>
> http://www.gnu.org/software/lightning/lightning.html
>
> it's a somewhat cross-platform library for generating and executing
> code at run-time.
>
> when I was looking at this stuff, someone had also suggested that
> rather than create a new runtime compiler, code could be converted to
> an existing one like Java....something everybody has access too and is
> rather mature. (I'm not sure if this is appropriate or not, but
> definitely worth some investigation.
>
>> There is a problem, I am hanging in understanding your code and getting
>> started with coding my self. I don`t have to much experience in coding
>> with gnu tools. (Never did big projects)
>
>
>> Do you know how I can debug .oct files? How I can trace into them with a
>> debugger?
>
>
> Ya. OK, I'm going to assume you have used gdb at least once, if you
> haven't feel free to speak up and I give you more. Basically the
> problem with .oct files is that they are dynamically linked, the
> implication is that symbols and code will not be available via the
> debugger until Octave has loaded that specific file. My way around
> this is to do the following, let's assume we're debugging the .oct
> file oregano.cc (the example included with Octave).
>
> 1. Start the debugger "gdb octave".
> 2. at the prompt  start octave using the "run" command
> 3. Once at the Octave command prompt you need to "coax" Octave into
> loading the command, but if we're trying to debug something it might
> not be desirable to actually execute the code, so I just type "help
> oregano". This forces Octave to load the file into memory.
> 4. Now that oregano.oct has been loaded, do a "control-c". This should
> drop you back to the debugger prompt where you should have access to
> the symbols and code in your .oct file. At this point you would
> presumably set breakpoint(s) within your code and continue down the
> debugging path.
>
> NOTE: A couple of caveauts to debugging Octave code. 1) Octave source
> makes have use  of inlined functions, so often when you arrive at a
> breakpoints which lies within your code. When stepping through code I
> have written, it hasn't been uncommon that I see none of my original
> code. 2) Compiler optimizations will further obfuscate your code. For
> example using "-g -O2" is sometimes confusing to step through.
>
>> What development environment do you use? I'm using Emacs (due
>> to my former small projects) - but  I'm still not convinced, its the
>> best decision.
>
>
> Well, I wouldn't call emacs a development environment, but I do use it
> for all my coding. I like the fact that it works on almost every
> platform I use and everything is accessible from a combination of keys
> (dont let that scare you off). Plus it's the right price. There are
> more "complete" development environments, but I just use a combination
> of emacs and gdb in one of three forms (command line gdb, emacs gdb or
> DDD which is a really nice graphical interface to gdb.) If you're used
> to a VC++ development environment going to emacs may have a shock
> factor since there are so many pictures.
>
> If you would like to work to together on your idea, I'd be more than
> happy to...perhaps we could turn something out that really gets people
> going.
>
> JD
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Octave Compiler

JD Cole
In reply to this post by JD Cole
Jens Ruecknagel wrote:

> Hi again JD Cole,

Please call me JD.

> after a lot of thinking I've concluded, that I want to build the
> compiler as an .oct file - but it should produce stand alone code
> (maybe based on liboctave) - but I have to check with my Professor,
> and I just cannot get in touch with him.

Probably the best choice, initially. This will be much more straight
forward than developing your own code to deal with all of Octave's
dynamic allocation, to say the least.

> So right now I am looking at your code. And now I have many Questions:
>
> What is the file goal.cc for?
> Is it just, what you wish compile to produce?

Yes, I believe that what it was.

>
> What does the operator |= in compile.cc do?

> Is it bitwise OR and assign? Or is it logical OR and assign (sorry
> about the C++ stuff, but obviously you thought about it)

I wouldn't go so far as to say I thought about it, but I have attached a
version of compile.cc with more comments, sorry, at the time I wasn't
thinking this code would get outside of my little sandbox.

>
>
> More Questions as son as I ran into them and cannot figure it out.

Something else that will end up being a big issues when you go to
implement the converter is how you are going to deal with references
external to the function your converting. Theses will include both .m
and .oct files.

1) Since .oct files are already in a useable state, as far as C++ is
concerned, you could possibly link your code with them (although I'm not
sure if code compiled as a shared object can later be linked into a  
single binary. So you could:
    a) A just have the compiler generate a list of modules it will be
dependent on in order to execute, and then copy or require those modules
to be copied into the same directory as the executable. (This would, of
course require you to familiarize yourself with loading shared
object...luckily Octaves designers have a nice C++ interface for shared
object loading.)
    b) Recompile those modules into your code, somewhat like octave does
with built in functions...
    c) There's always a "c", but I'm not sure what it is.
    d) Don't allow compiled functions to contain external
references....with the exeception of builtins, which I think are in one
of the octave libraries. (This may be the best idea as a first attempt
at code conversion...it's not as glorious, but definitely has value.)
    e).....

2) .m files are another issues, which could end up being somewhat
recursive should you decide to compile the .m's external to your code.
I'm talking about when you determine that the external function is a .m
not a .oct file, you choose to compile it into your code also.

Anyways, more to think about. Anyone have any comments on this?

JD