Re: speed of octave interpreter

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: speed of octave interpreter

Paul Kienzle
[redirected to maintainers list]

I'm pretty sure we don't want to reload a function definition
haphazardly during a long running computation.

Maybe should only check the timestamp at a few select points, such as
the first function call after input() or after returning to the top
level, and provide the user with some way to force it?

To be more specific, something like the following:

   try_reload(function)
     if nargin > 0
       if function.timestamp > function.loadtime
          function.reload()
          function.loadtime = now()
     else
          global reset_time
       reset_time = now()
     endif

   feval()
     ...
     global reset_time
     if function.loadtime < reset_time && function.timestamp >
function.loadtime
       function.reload()
       function.loadtime = now()
     endif
     ...

Then input() and top_level() would have calls to try_reload(), and the
user could call try_reload(fn) to check if a specific function has
changed, or try_reload() to check everything.  A GUI event loop would
probably also call try_reload() whenever it goes idle.

Sorry, no patch.

- Paul

On Sep 25, 2005, at 7:58 PM, Brian Blais wrote:

> Stefan van der Walt wrote:
>> On Sun, Sep 25, 2005 at 01:35:23PM -0400, Brian Blais wrote:
>>> I am running Octave (2.1.71) on Linux (SuSe 9.1), and have written a
>>> simple recursive minimax example for a class, and it seemed to run
>>> very
>>> slowly.   So, on a whim, I decided to test it in Scilab 3.0, Matlab
>>> 7.0
>> Since you are dabbling with the murky art of recursion, you probably
>> want to set
>> ignore_function_time_stamp = "all"
>
> Holy smokes!  What a difference!
>
>>> ignore_function_time_stamp = "system";
>>> tic; nim_minimax(15); toc
> ans = 56.552
>>> ignore_function_time_stamp = "all";
>>> tic; nim_minimax(15); toc
> ans = 1.4262
>
>
> thanks,
>
>
> bb
>
>
> --
> -----------------
>
>             [hidden email]
>             http://web.bryant.edu/~bblais
>
>
>
> -------------------------------------------------------------
> Octave is freely available under the terms of the GNU GPL.
>
> Octave's home on the web:  http://www.octave.org
> How to fund new projects:  http://www.octave.org/funding.html
> Subscription information:  http://www.octave.org/archive.html
> -------------------------------------------------------------
>

Reply | Threaded
Open this post in threaded view
|

Re: speed of octave interpreter

John W. Eaton-6
On 25-Sep-2005, Paul Kienzle wrote:

| [redirected to maintainers list]
|
| I'm pretty sure we don't want to reload a function definition
| haphazardly during a long running computation.
|
| Maybe should only check the timestamp at a few select points, such as
| the first function call after input() or after returning to the top
| level, and provide the user with some way to force it?
|
| To be more specific, something like the following:
|
|    try_reload(function)
|      if nargin > 0
|        if function.timestamp > function.loadtime
|           function.reload()
|           function.loadtime = now()
|      else
|  global reset_time
|        reset_time = now()
|      endif
|
|    feval()
|      ...
|      global reset_time
|      if function.loadtime < reset_time && function.timestamp >
| function.loadtime
|        function.reload()
|        function.loadtime = now()
|      endif
|      ...
|
| Then input() and top_level() would have calls to try_reload(), and the
| user could call try_reload(fn) to check if a specific function has
| changed, or try_reload() to check everything.  A GUI event loop would
| probably also call try_reload() whenever it goes idle.
|
| Sorry, no patch.

Currently, each time Octave looks up the definition of a symbol, it
tries to make sure that it has the appropriate definition.  If the
symbol is already defined as a function, then we check to see if there
is a newer definition available and if so, load the newer version of
the file.  The check is only supposed to happen once between
prompts for user input.  You can also turn off the check completely if
you set ignore_function_timestamps.  But getting to the point of
deciding whether we can skip the check requires several function
calls.  See the function symbol_out_of_date in

Changing to the method you outline above would probably be an
improvement unless someone had an extremely large number of functions
loaded in the symbol table but was only using a small number of them.
It should not be too hard to implmement.  We would need to write a
method for the symbol_table class that calls symbol_out_of_date on
every symbol_record in the table that is defined as a function.  Then
call this method on the fbi_sym_tab at the point of prompting for user
input.  That should be sufficient for now because all function
definitions are stored in the same global symbol table.  We would need
to do something else if we modify Octave to store the definitions of
nested subfunctions in the symbol table associated with the parent
function.

FWIW, the code that handles all of this has been around since nearly
the beginning of Octave.  It's been hacked at in various ways but
never carefully examined and (re)designed.  So not surprisingly, there
are currently some other bugs as well.  One is that a cd command can
cause subfunction lookups to return the wrong function definition.
Another is that timestamps of fuctions called through function handles
are not checked.  Both of these were reported recently but don't seem
simple to fix.  It may be time for a thorough review of the way this
mess is implemented.

jwe

Reply | Threaded
Open this post in threaded view
|

Re: speed of octave interpreter

Paul Kienzle

On Sep 26, 2005, at 10:56 AM, John W. Eaton wrote:

> On 25-Sep-2005, Paul Kienzle wrote:
>
> | [redirected to maintainers list]
> |
> | I'm pretty sure we don't want to reload a function definition
> | haphazardly during a long running computation.
> |
> | Maybe should only check the timestamp at a few select points, such as
> | the first function call after input() or after returning to the top
> | level, and provide the user with some way to force it?
> |
> | To be more specific, something like the following:
> |
> |    try_reload(function)
> |      if nargin > 0
> |        if function.timestamp > function.loadtime
> |           function.reload()
> |           function.loadtime = now()
> |      else
> |  global reset_time
> |        reset_time = now()
> |      endif
> |
> |    feval()
> |      ...
> |      global reset_time
> |      if function.loadtime < reset_time && function.timestamp >
> | function.loadtime
> |        function.reload()
> |        function.loadtime = now()
> |      endif
> |      ...
> |
> | Then input() and top_level() would have calls to try_reload(), and
> the
> | user could call try_reload(fn) to check if a specific function has
> | changed, or try_reload() to check everything.  A GUI event loop would
> | probably also call try_reload() whenever it goes idle.
> |
> | Sorry, no patch.
>
> Currently, each time Octave looks up the definition of a symbol, it
> tries to make sure that it has the appropriate definition.  If the
> symbol is already defined as a function, then we check to see if there
> is a newer definition available and if so, load the newer version of
> the file.  The check is only supposed to happen once between
> prompts for user input.  You can also turn off the check completely if
> you set ignore_function_timestamps.  But getting to the point of
> deciding whether we can skip the check requires several function
> calls.  See the function symbol_out_of_date in

I think this is what I was proposing.

I would be interested to know if the performance on mingw is as
bad as on cygwin.  Things like make are slow on cygwin compared
to the same machine on linux, and I think that it is because of
the file subsystem.

Regardless, that doesn't explain the enormous time differences
observed with ignore_function_timestamp.

> Changing to the method you outline above would probably be an
> improvement unless someone had an extremely large number of functions
> loaded in the symbol table but was only using a small number of them.
> It should not be too hard to implmement.  We would need to write a
> method for the symbol_table class that calls symbol_out_of_date on
> every symbol_record in the table that is defined as a function.  Then
> call this method on the fbi_sym_tab at the point of prompting for user
> input.  That should be sufficient for now because all function
> definitions are stored in the same global symbol table.  We would need
> to do something else if we modify Octave to store the definitions of
> nested subfunctions in the symbol table associated with the parent
> function.

Checking timestamps for rarely used functions doesn't
sound like a good idea to me if the problem is with
the speed of cygwin file accesses.  I would want to
check the timestamp only if necessary.

> FWIW, the code that handles all of this has been around since nearly
> the beginning of Octave.  It's been hacked at in various ways but
> never carefully examined and (re)designed.  So not surprisingly, there
> are currently some other bugs as well.  One is that a cd command can
> cause subfunction lookups to return the wrong function definition.
> Another is that timestamps of fuctions called through function handles
> are not checked.  Both of these were reported recently but don't seem
> simple to fix.  It may be time for a thorough review of the way this
> mess is implemented.

I had trouble navigating through it and doing what I wanted.  With the
new features such as local functions and dispatch, I'm sure the code
base would benefit from refactoring.  Also, this would make it easier
to implement mex and maybe a foreign function interface.

- Paul

Reply | Threaded
Open this post in threaded view
|

Re: speed of octave interpreter

John W. Eaton-6
On 26-Sep-2005, Paul Kienzle wrote:

| I would be interested to know if the performance on mingw is as
| bad as on cygwin.  Things like make are slow on cygwin compared
| to the same machine on linux, and I think that it is because of
| the file subsystem.

Part of it may be the filesystem.  I think the other thing is the cost
of fork.  From a quick look at the sources, it seems that GNU Make on
Windows uses CreateProcess instead of fork/exec.  I don't see any
special code for Cygwin, so I assume it just does the Unixy thing and
calls fork followed by exec.  On Cygwin, I think this copies the
entire parent process even when the next action is exec (making the
copy unnecessary)..

| Regardless, that doesn't explain the enormous time differences
| observed with ignore_function_timestamp.

I think that was a bug.  I don't see such a dramatic difference with
the current sources (either 2.9.x or the current 2.1.x).  But I also
don't see anything in the diffs between the current 2.1.x sources and
2.1.71 that would account for the difference, so I'm trying to verify
that the bug really is fixed.  I do see the problem in the copy of
2.1.69 that is distributed with Debian testing.

When I run

  strace -o foo.out octave
  ...
  octave> nim_minimax (15)

with 2.1.69, I see an insanely large number of calls to "access" that
should not be happening.

jwe