Re: ROctave

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: ROctave

John W. Eaton-6
[I'm copying this to the octave-maintainers mailing list so that we
can maybe start a discussion of this there as well.  --jwe]

On  7-Dec-2002, Duncan Temple Lang <[hidden email]> wrote:

| The only difficulty that might arise in the longer term is what I
| would ideally like to have: an Octave embedded in R.

OK, I've been thinking about how to do something similar (allow people
to load Octave in other programs) and it is currently possible to load
all of Octave as a shared library.  The main Octave program is now
just

  #ifdef HAVE_CONFIG_H
  #include <config.h>
  #endif

  #include "f77-fcn.h"
  #include "lo-ieee.h"

  #include "octave.h"

  int
  main (int argc, char **argv)
  {
    return octave_main (argc, argv);
  }

and octave_main and everything else is in a set of shared libraries,
so you can load them and call octave_main in your own application.
The problem is that this also starts up the interactive command loop.
So we really need an interface that looks something like

  // load shared libraries...

  octave_init ();  // perhaps with an argument to say not interactive?

  // other stuff...

  for (;;)
    {
       // ...

       octave_value result = parse_and_execute_octave_command (???);

       // or some other method of controlling Octave...
    }

Though I'm not sure what would be best.  As things are now,
octave_value is probably only useful inside a C++ program, and it's
not clear to me how we should control Octave from another program.  In
some cases, parsing and evaluating strings might be OK, but in other
cases, you may want to build a parse tree using some function calls
and then evaluate that directly, skipping the lexer/parser.

As a minimum, I think we need to write a few functions that allow us
to initialize Octave without starting up the interactive I/O system so
we can just use Octave as a compute engine.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: ROctave

Andy Adler
On Mon, 16 Dec 2002, John W. Eaton wrote:

>   octave_init ();  // perhaps with an argument to say not interactive?
>
>   // other stuff...
>
>   for (;;)
>     {
>        // ...
>
>        octave_value result = parse_and_execute_octave_command (???);
>
>        // or some other method of controlling Octave...
>     }
>
> Though I'm not sure what would be best.  As things are now,
> octave_value is probably only useful inside a C++ program, and it's
> not clear to me how we should control Octave from another program.  In
> some cases, parsing and evaluating strings might be OK, but in other
> cases, you may want to build a parse tree using some function calls
> and then evaluate that directly, skipping the lexer/parser.
>
> As a minimum, I think we need to write a few functions that allow us
> to initialize Octave without starting up the interactive I/O system so
> we can just use Octave as a compute engine.

This is exactly the functionality I need to write a proper
Inline::Octave for Perl. Currently I'm doing it by controlling
stdin / stdout with two pipes, but it's a real pain.

Note: this API also needs a good way to get errors and
warnings back from octave.

Andy


Reply | Threaded
Open this post in threaded view
|

Re: ROctave

Paul Kienzle
----- Original Message -----
From: "Andy Adler" <[hidden email]>
To: "John W. Eaton" <[hidden email]>
Cc: "Duncan Temple Lang" <[hidden email]>;
"octave-maintainers mailing list" <[hidden email]>
Sent: Monday, December 16, 2002 1:31 PM
Subject: Re: ROctave


> On Mon, 16 Dec 2002, John W. Eaton wrote:
> >   octave_init ();  // perhaps with an argument to say not interactive?
> >
> >   // other stuff...
> >
> >   for (;;)
> >     {
> >        // ...
> >
> >        octave_value result = parse_and_execute_octave_command (???);
> >
> >        // or some other method of controlling Octave...
> >     }
> >
> > Though I'm not sure what would be best.  As things are now,
> > octave_value is probably only useful inside a C++ program, and it's
> > not clear to me how we should control Octave from another program.  In
> > some cases, parsing and evaluating strings might be OK, but in other
> > cases, you may want to build a parse tree using some function calls
> > and then evaluate that directly, skipping the lexer/parser.
> >
> > As a minimum, I think we need to write a few functions that allow us
> > to initialize Octave without starting up the interactive I/O system so
> > we can just use Octave as a compute engine.

Are there cases where we would not want to initialize everything?  E.g., the
types but not the symbol tables?  Probably not since this interface is
designed to run interpreted code, so the whole interpreter needs to be
initialized.

>
> This is exactly the functionality I need to write a proper
> Inline::Octave for Perl. Currently I'm doing it by controlling
> stdin / stdout with two pipes, but it's a real pain.

Agreed.  It is especially difficult to protect against poor code being sent
to
the interpreter.  I have much better luck in my socket listening code
because the eval statement sets a natural boundary within which all blocks
must be closed.

>
> Note: this API also needs a good way to get errors and
> warnings back from octave.

Presumably we need to turn on octave's signal handling upon entering eval
and turn it off again upon leaving.

We also need to figure out what to do with the input statement.

And capture the output so that it can be returned to the caller.

What happens when the debugger is triggered?

Another feature I would like is an "eval in" function.  I find in my code
that I have a lot of interplay between tcl and octave.  I'll intersperse a
few lines of octave with a few lines of tcl throughout a subroutine. The
problem that I'm having is that temporary variables from one subroutine are
colliding with temporary variables in other subroutines.  To avoid this, I
want to be able to create a symbol table and call eval with it a bunch of
times, then possibly empty or delete it.

>
> Andy
>


Reply | Threaded
Open this post in threaded view
|

Re: ROctave

Duncan Temple Lang-2
In reply to this post by John W. Eaton-6

Hi John.

I finally got a chance to get to this and put Octave inside R.  Thanks
for making those changes. It is a great start and I made some trivial
additions to conditionally activate different parts of the code. I've
attached the patch file for the 3 files that were modified.  With
those changes compiled into liboctave and calling octave_main() with
the new 3rd argument - embedded - as 1, I get the Octave engine
without the octave event/input loop.

We could, of course, also split the octave_main into more pieces, some
common and some specific to interactive use, and have main() be
slightly more complex. Either works well for me; more routines gives
greater customizability at the expense of complexity. So sticking with
arguments to octave_main() might be simplest, and that is what I was
going for.  If we also want to allow conditional startup of the I/O
system, etc.  then we may want a general options object or
split the material into multiple routines.

Regarding whether one wants to parse and evaluate strings or build
parse trees, all our inter-system interfaces typically work by calling
regular functions in the embedded system.  So for Octave in R, the
primary function is .Octave which takes the name of the Octave
function to call and an arbitrary number of arguments.  I convert
those R values to Octave objects (octave_values) and call feval() and
convert the result back to R.  We try to dissuade people from using
commands passed as strings as then they have to know two languages.
And one cannot express all types of expressions, e.g. when one needs a
reference to an object in one system.

Cheers,
 D.


John W. Eaton wrote:

> [I'm copying this to the octave-maintainers mailing list so that we
> can maybe start a discussion of this there as well.  --jwe]
>
> On  7-Dec-2002, Duncan Temple Lang <[hidden email]> wrote:
>
> | The only difficulty that might arise in the longer term is what I
> | would ideally like to have: an Octave embedded in R.
>
> OK, I've been thinking about how to do something similar (allow people
> to load Octave in other programs) and it is currently possible to load
> all of Octave as a shared library.  The main Octave program is now
> just
>
>   #ifdef HAVE_CONFIG_H
>   #include <config.h>
>   #endif
>
>   #include "f77-fcn.h"
>   #include "lo-ieee.h"
>
>   #include "octave.h"
>
>   int
>   main (int argc, char **argv)
>   {
>     return octave_main (argc, argv);
>   }
>
> and octave_main and everything else is in a set of shared libraries,
> so you can load them and call octave_main in your own application.
> The problem is that this also starts up the interactive command loop.
> So we really need an interface that looks something like
>
>   // load shared libraries...
>
>   octave_init ();  // perhaps with an argument to say not interactive?
>
>   // other stuff...
>
>   for (;;)
>     {
>        // ...
>
>        octave_value result = parse_and_execute_octave_command (???);
>
>        // or some other method of controlling Octave...
>     }
>
> Though I'm not sure what would be best.  As things are now,
> octave_value is probably only useful inside a C++ program, and it's
> not clear to me how we should control Octave from another program.  In
> some cases, parsing and evaluating strings might be OK, but in other
> cases, you may want to build a parse tree using some function calls
> and then evaluate that directly, skipping the lexer/parser.
>
> As a minimum, I think we need to write a few functions that allow us
> to initialize Octave without starting up the interactive I/O system so
> we can just use Octave as a compute engine.
>
> jwe

PATCHES (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ROctave

John W. Eaton-6
On 10-Jan-2003, Duncan Temple Lang <[hidden email]> wrote:

| I finally got a chance to get to this and put Octave inside R.  Thanks
| for making those changes. It is a great start and I made some trivial
| additions to conditionally activate different parts of the code. I've
| attached the patch file for the 3 files that were modified.  With
| those changes compiled into liboctave and calling octave_main() with
| the new 3rd argument - embedded - as 1, I get the Octave engine
| without the octave event/input loop.

OK.

| We could, of course, also split the octave_main into more pieces, some
| common and some specific to interactive use, and have main() be
| slightly more complex. Either works well for me; more routines gives
| greater customizability at the expense of complexity. So sticking with
| arguments to octave_main() might be simplest, and that is what I was
| going for.  If we also want to allow conditional startup of the I/O
| system, etc.  then we may want a general options object or
| split the material into multiple routines.

For now, I don't think it matters whether it is one function with an
argument or a set of functions, though we may want that later.

In your patch, you have

   if(!embedded)
      install_signal_handlers ();

which I think is a problem.  If you avoid setting up signal handlers
for Octave, then what happens when an interrupt signal is received
while the Octave interpreter is running?  Currently, Octave's SIGINT
handler sets a flag and returns, allowing execution to continue until
it is safe to throw a C++ exception that will return control to the
top-level event loop.  Doing that ensures that destructors for objects
on the stack are properly destroyed.  If you skip that by not
installing Octave's signal handlers, then you are asking for memory
leaks or even possibly leaving some object in an inconsistent state,
which could cause unpredictible results on later calls.

It seems that if octave_main is called with embedded = 1, we still
need to initialize the signal handlers, but do something different
when a signal is encountered.  Something like

  save original signal handler
  set up octave signal handler

and then when a signal happens:

  perform octave cleanup
  restore original signal handler
  raise signal again, so the orignal handler can catch it

This would probably need to happen on each call to the Octave
interpreter, and each function that the embeddable Octave exports
would need to be wrapped by a try-catch block to allow it to catch an
octave_interrupt exception.  This should work fine if we only have a
few functions to export.

Or am I missing something?

jwe


Reply | Threaded
Open this post in threaded view
|

Re: ROctave

Duncan Temple Lang-2
John W. Eaton wrote:
> On 10-Jan-2003, Duncan Temple Lang <[hidden email]> wrote:
>

> In your patch, you have
>
>    if(!embedded)
>       install_signal_handlers ();
>
> which I think is a problem.  If you avoid setting up signal handlers
> for Octave, then what happens when an interrupt signal is received
> while the Octave interpreter is running?  Currently, Octave's SIGINT
> handler sets a flag and returns, allowing execution to continue until
> it is safe to throw a C++ exception that will return control to the
> top-level event loop.  Doing that ensures that destructors for objects
> on the stack are properly destroyed.  If you skip that by not
> installing Octave's signal handlers, then you are asking for memory
> leaks or even possibly leaving some object in an inconsistent state,
> which could cause unpredictible results on later calls.

Yep, this needs to be worked out carefully and I went for the safe way
of getting control back to the host application (R in this case)
unconditionally. But in all the other inter-system interfaces we have
done, exception handling and general control flow in the case of
errors is the thing that needs the most technical attention to the
specifics. When we embed R in other things, we use a R_tryEval() which
guarantees to return to that spot, not the event loop since that may
not be activated.

>
> It seems that if octave_main is called with embedded = 1, we still
> need to initialize the signal handlers, but do something different
> when a signal is encountered.  Something like
>
>   save original signal handler
>   set up octave signal handler
>
> and then when a signal happens:
>
>   perform octave cleanup
>   restore original signal handler
>   raise signal again, so the orignal handler can catch it
>
> This would probably need to happen on each call to the Octave
> interpreter, and each function that the embeddable Octave exports
> would need to be wrapped by a try-catch block to allow it to catch an
> octave_interrupt exception.  This should work fine if we only have a
> few functions to export.


Yep, that's what we do in R and I think should work just fine.  We
have the R_tryEval() or else one can establish a customized "CONTEXT"
which will trap the jumps and allow one to handle these in a special
way. So definitely we need to do something along these lines for Octave.
As I mention, I just wanted to ensure we didn't throw the exception
and have nobody catch it or jump to the octave loop that wasn't actually
running. So hopefully that explains why I put the if(!embedded) there.


Thanks,
  D.

>
> Or am I missing something?
>
> jwe