Re: Crash with inline

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

Teemu Ikonen
On 15/09/04 16:30, John W. Eaton wrote:
> On 15-Sep-2004, Teemu Ikonen <[hidden email]> wrote:
> | While trying to achieve lambda-like functionality with inline, I ran into
> | this crasher:
> Please try the following patch.

Thanks for the quick reply.

Now, an obligatory stupid question: Since Octave already seems to have
functions as first-class objects, would it be hard to implement real closures?

I'm not asking for someone to actually do this any time soon, I'd just like
to know.

Also, I'm wondering about the difference between inline functions and user
defined functions defined on the command line with the function keyword.
Inline functions are listed as variables in the who output and can be saved
at least in the octave binary format. Are there any deeper differences? Is
there any way of saving functions defined by the function keyword?

Teemu


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

David Bateman-3
> Also, I'm wondering about the difference between inline functions and user
> defined functions defined on the command line with the function keyword.

An inline function deosn't appear in the fbi_sym_tab. Apart from that
not much different. A function handle however has a subtle difference,
for example

octave:1> fcn = @not
octave:2> function y = not (x), y = x; endfunction
octave:3> fcn(1), not(1)
ans = 0
ans = 1

This happens due to the fact that the function handle saves the
octave_function in the original symbol table before it was replaced..
However, this is not true for inline functions, for example

octave:1> fcn = inline('not(x)')
fcn =

f(x) = not(x)

octave:2> not = inline('x')
not =

f(x) = x

octave:3> fcn(1), not(1)
ans = 0
ans = 0

Err, in fact this seems to be a compatiability problem with matlab,
as it seems matlab save a copy of the current symbol table for inline
functions, since in matlab the above example gives

>> fcn = inline('not(x)')

fcn =

     Inline function:
     fcn(x) = not(x)

>> not = inline('x')

not =

     Inline function:
     not(x) = x

>> fcn(1), not(1)

ans =

     0


ans =

     1

This probably also points to a deeper compatiability issue with function
handles as functions called by the original function might also change
in the current symbol table...

> Inline functions are listed as variables in the who output and can be saved
> at least in the octave binary format. Are there any deeper differences? Is
> there any way of saving functions defined by the function keyword?

You are upto date, that was only in the CVS a couple of days ago... Function
handles and inline functions can be saved in hdf5, octave binary and ascii
formats. I don't think it makes sense to save user functions, since when
saving all variables you'd effective resave all of your script files.
Perhaps functions that were entered interactively should be flagged and
saved in this case.

Regards
D.

--
David Bateman                                [hidden email]
Motorola CRM                                 +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax)
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as:

[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

John W. Eaton-6
In reply to this post by Teemu Ikonen
On 16-Sep-2004, Teemu Ikonen <[hidden email]> wrote:

| Now, an obligatory stupid question: Since Octave already seems to have
| functions as first-class objects,

They are not really first-class objects, are they?  If you write

  function f ... end

  g = f;

then g is not a function, but the result of calling the function.
Similarly, if you write

  f

you don't see the definition of the function (or some other message
saying, "f is a function") but instead, the function is called.  So
you can't directly pass functions around like any other data object.

Rather than fix this by saying "if you want to call a function, you
have to use "f (...)"; otherwise, you are referring to the function
object itself", the developers of the other leading brand decided to
invent "function handles".

| would it be hard to implement real closures?

I'm not sure.  How would this work within the current language?

| Also, I'm wondering about the difference between inline functions and user
| defined functions defined on the command line with the function keyword.
| Inline functions are listed as variables in the who output and can be saved
| at least in the octave binary format. Are there any deeper differences? Is
| there any way of saving functions defined by the function keyword?

We also have anonymous function handles.  Given those, it seems like
inline functions are not really necessary.

In Octave, inline functions are a special kind of function handle.

If you want to save a function that you've defined at the command
line, you can do this:

  function foo ... end

  f = fopen ("foo.m", "w");
  fprintf (f, "%s", type ("foo"));
  fclose (f);

If you've saved it somewhere in Octave's LOADPATH, then you can just
call it to reload it.  Or, you can do

  source ("foo.m")

or you can write

  f = fopen ("foo.m", "r");
  t = setstr (fread (f)');
  fclose (f);
  eval (t);

jwe

 


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

Teemu Ikonen
In reply to this post by Teemu Ikonen
On 16/09/04 11:01, John W. Eaton wrote:
> | functions as first-class objects,
> They are not really first-class objects, are they?  If you write

Oops, indeed I mistook the function handles returned by inline as functions.

> | would it be hard to implement real closures?
> I'm not sure.  How would this work within the current language?

Well, functions defined in the command line either by inline or function
keyword would find undefined variables from the top-level environment (if
they exist) and store their value. Something like:

> a = 1;
> f = inline("x + a", "x");
> f(1)
ans = 2
> a = pi;
> f(1)
ans = 2

Looks straightforward, but there might be all kinds of subtleties I'm not
aware of...

Teemu


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

David Bateman-3
According to Teemu Ikonen <[hidden email]> (on 09/16/04):

> On 16/09/04 11:01, John W. Eaton wrote:
> > | functions as first-class objects,
> > They are not really first-class objects, are they?  If you write
>
> Oops, indeed I mistook the function handles returned by inline as functions.
>
> > | would it be hard to implement real closures?
> > I'm not sure.  How would this work within the current language?
>
> Well, functions defined in the command line either by inline or function
> keyword would find undefined variables from the top-level environment (if
> they exist) and store their value. Something like:
>
> > a = 1;
> > f = inline("x + a", "x");
> > f(1)
> ans = 2
> > a = pi;
> > f(1)
> ans = 2
>
> Looks straightforward, but there might be all kinds of subtleties I'm not
> aware of...

Basically to do this we'd need a copy constructor for symbol_table and make
a copy of curr_sym_tab when creating the function handle. As inline functions
are just specialized function handles and inherit the methods of function
handles no special treatment would be needed.

The problem then arises in a case where for example you have a function
on the loadpath, create a function handle that depends on this function,
and then change directories. If the function hasn't already been parsed
when the copy of curr_sym_tab is made, and a different version of this
function exists in the need directory, then which version is called. So
you also need to save the loadpath.

Ok, lets get even more paranoid. With the same case as above without
changing directories. However, the function that the function handle
depends on is edited... Should the function handle still use the old
definition or the new one.

So the question is exactly how much of the context is saved when creating
a function handle.. If it is everything, creating a function handle might
be a very very expensive operation...

I haven't checked how matlab does this, but I'd thing a reasonable solution
would be to save curr_sym_tab, fbi_sym_tab and the loadpath. However, it
would be good to hear if others have other ideas..

D.

--
David Bateman                                [hidden email]
Motorola CRM                                 +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax)
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as:

[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

David Bateman-3
This made me check how matlab/octave handle different versions
of the function when changing directory. Consider the matlab
example

>> test(1);
>> which test
/home/dbateman/tmp/test.m
>> cd ..
>> which test
/home/dbateman/test.m

Doing the same in octave gives

octaveL1> test(1);
octave:2> which test
test is the user-defined function from the file
/home/dbateman/tmp/test.m
octave:3> cd ..
octave:4> which test
test is the user-defined function from the file
/home/dbateman/tmp/test.m

So it seems that octave keeps the compiled version from its symbol
table even after the directory change, while matlab picks up the
fact that there is a new version in the loadpath. What I suspect
that matlab does is that when doing a cd it invalidates all of the
functions in its symbol table that are located elsewhere it its
loadpath after the directory change...

Should octave do the same thing?

Regards
David

--
David Bateman                                [hidden email]
Motorola CRM                                 +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax)
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as:

[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

John W. Eaton-6
On 16-Sep-2004, David Bateman <[hidden email]> wrote:

| So it seems that octave keeps the compiled version from its symbol
| table even after the directory change, while matlab picks up the
| fact that there is a new version in the loadpath. What I suspect
| that matlab does is that when doing a cd it invalidates all of the
| functions in its symbol table that are located elsewhere it its
| loadpath after the directory change...

I'd guess that CD is not doing anything special.  Instead, WHICH
is probably looking up what symbol should be found, and instead of
assuming that a symbol in memory must be the correct one, is looking
at the load path.  If we decide to fix this in Octave, then I think we
need to change the way symbol_out_of_data works.  Instead of just
checking symbol timestamps, we need to see if the function found in
the path is different from the one found in the current symbol table.
Then the which command may also need to be changed to check for out of
date symbols (probably instead of duplicating any code or logic, it
needs to use the same method of looking up a symbol that the
interpreter uses when attempting to evaluate a symbol).

Now, what to do about functions with the same name defined on the
command line?  Should those always be favored over what is found in
the load path?

jwe



Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

John W. Eaton-6
In reply to this post by Teemu Ikonen
On 16-Sep-2004, Teemu Ikonen <[hidden email]> wrote:

| Well, functions defined in the command line either by inline or function
| keyword would find undefined variables from the top-level environment (if
| they exist) and store their value. Something like:
|
| > a = 1;
| > f = inline("x + a", "x");
| > f(1)
| ans = 2
| > a = pi;
| > f(1)
| ans = 2
|
| Looks straightforward, but there might be all kinds of subtleties I'm not
| aware of...

I'm not sure it is a good idea to modify the scoping rules this way.
Why should inline functions or functions defined at the command line
behave differently from functions defined in a file?  Seems messy to
me.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

John W. Eaton-6
In reply to this post by John W. Eaton-6
On 16-Sep-2004, I wrote:

| I'd guess that CD is not doing anything special.  Instead, WHICH
| is probably looking up what symbol should be found, and instead of
| assuming that a symbol in memory must be the correct one, is looking
| at the load path.  If we decide to fix this in Octave, then I think we
| need to change the way symbol_out_of_data works.  Instead of just
| checking symbol timestamps, we need to see if the function found in
| the path is different from the one found in the current symbol table.

I think the following patch will make Octave work in a compatible way.

| Then the which command may also need to be changed to check for out of
| date symbols (probably instead of duplicating any code or logic, it
| needs to use the same method of looking up a symbol that the
| interpreter uses when attempting to evaluate a symbol).

This part was already done.

| Now, what to do about functions with the same name defined on the
| command line?  Should those always be favored over what is found in
| the load path?

Currently we will always favor the function defined on the command
line.  Doing something like

  octave:1> function foo () ... end

and then

  which foo

or

  cd some-directory-with-a-foo.m-file
  which foo

will tell you that foo is a user defined function.

BTW, it would be nice if in this case WHICH would till us that it has
no corresponding file.  Likewise, for built-in functions, it would be
nice to have it say in what file it was originally defined.

jwe


2004-09-16  John W. Eaton  <[hidden email]>

        * variables.cc (symbol_out_of_date): Always look in LOADPATH.


Index: src/variables.cc
===================================================================
RCS file: /usr/local/cvsroot/octave/src/variables.cc,v
retrieving revision 1.260
diff -u -r1.260 variables.cc
--- a/src/variables.cc 11 Sep 2004 13:05:39 -0000 1.260
+++ b/src/variables.cc 16 Sep 2004 18:41:17 -0000
@@ -772,19 +772,28 @@
  {
   time_t tp = tmp->time_parsed ();
 
-  std::string fname;
+  std::string nm = tmp->name ();
 
-  if (tmp->is_dld_function ())
-    fname = ff;
-  else
-    fname = fcn_file_in_path (ff);
+  string_vector names (2);
 
-  tmp->mark_fcn_file_up_to_date (octave_time ());
+  names[0] = nm + ".oct";
+  names[1] = nm + ".m";
 
-  file_stat fs (fname);
+  std::string file = octave_env::make_absolute
+    (Vload_path_dir_path.find_first_of (names),
+     octave_env::getcwd ());
 
-  if (fs && fs.is_newer (tp))
+  if (file != ff)
     retval = true;
+  else
+    {
+      tmp->mark_fcn_file_up_to_date (octave_time ());
+
+      file_stat fs (ff);
+
+      if (fs && fs.is_newer (tp))
+ retval = true;
+    }
  }
     }
  }


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

David Bateman-3
Ok this patch made the function that was picked up change after a CD,
but WHICH still identifies the incorrect function, even after it is
reparsed (ie. used).

On another point what about the issue raised in

http://www.octave.org/mailing-lists/octave-maintainers/2004/701

for inline functions. Should octave be saving the curr_sym_tab, fbi_sym_tab,
loadpath as context for the function handles? Can I implement copy functions
for symbol_table, which would be needed for this? Or is there a reason why
such a copy function should be avoided?

Regards
David

According to John W. Eaton <[hidden email]> (on 09/16/04):

> On 16-Sep-2004, I wrote:
>
> | I'd guess that CD is not doing anything special.  Instead, WHICH
> | is probably looking up what symbol should be found, and instead of
> | assuming that a symbol in memory must be the correct one, is looking
> | at the load path.  If we decide to fix this in Octave, then I think we
> | need to change the way symbol_out_of_data works.  Instead of just
> | checking symbol timestamps, we need to see if the function found in
> | the path is different from the one found in the current symbol table.
>
> I think the following patch will make Octave work in a compatible way.
>
> | Then the which command may also need to be changed to check for out of
> | date symbols (probably instead of duplicating any code or logic, it
> | needs to use the same method of looking up a symbol that the
> | interpreter uses when attempting to evaluate a symbol).
>
> This part was already done.
>
> | Now, what to do about functions with the same name defined on the
> | command line?  Should those always be favored over what is found in
> | the load path?
>
> Currently we will always favor the function defined on the command
> line.  Doing something like
>
>   octave:1> function foo () ... end
>
> and then
>
>   which foo
>
> or
>
>   cd some-directory-with-a-foo.m-file
>   which foo
>
> will tell you that foo is a user defined function.
>
> BTW, it would be nice if in this case WHICH would till us that it has
> no corresponding file.  Likewise, for built-in functions, it would be
> nice to have it say in what file it was originally defined.
>
> jwe
>
>
> 2004-09-16  John W. Eaton  <[hidden email]>
>
> * variables.cc (symbol_out_of_date): Always look in LOADPATH.
>
>
> Index: src/variables.cc
> ===================================================================
> RCS file: /usr/local/cvsroot/octave/src/variables.cc,v
> retrieving revision 1.260
> diff -u -r1.260 variables.cc
> --- a/src/variables.cc 11 Sep 2004 13:05:39 -0000 1.260
> +++ b/src/variables.cc 16 Sep 2004 18:41:17 -0000
> @@ -772,19 +772,28 @@
>   {
>    time_t tp = tmp->time_parsed ();
>  
> -  std::string fname;
> +  std::string nm = tmp->name ();
>  
> -  if (tmp->is_dld_function ())
> -    fname = ff;
> -  else
> -    fname = fcn_file_in_path (ff);
> +  string_vector names (2);
>  
> -  tmp->mark_fcn_file_up_to_date (octave_time ());
> +  names[0] = nm + ".oct";
> +  names[1] = nm + ".m";
>  
> -  file_stat fs (fname);
> +  std::string file = octave_env::make_absolute
> +    (Vload_path_dir_path.find_first_of (names),
> +     octave_env::getcwd ());
>  
> -  if (fs && fs.is_newer (tp))
> +  if (file != ff)
>      retval = true;
> +  else
> +    {
> +      tmp->mark_fcn_file_up_to_date (octave_time ());
> +
> +      file_stat fs (ff);
> +
> +      if (fs && fs.is_newer (tp))
> + retval = true;
> +    }
>   }
>      }
>   }

--
David Bateman                                [hidden email]
Motorola CRM                                 +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax)
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as:

[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

John W. Eaton-6
On 16-Sep-2004, David Bateman <[hidden email]> wrote:

| Ok this patch made the function that was picked up change after a CD,
| but WHICH still identifies the incorrect function, even after it is
| reparsed (ie. used).

Can you show precisely what the problem is?  Here is what I see:

  octave:1> foo
  ans = 1
  octave:2> which foo
  foo is the user-defined function from the file
  /scratch/jwe/build/octave/foo.m
  octave:3> cd ..
  octave:4> which foo
  foo is the user-defined function from the file
  /scratch/jwe/build/foo.m
  octave:5> foo
  ans = 2
  octave:6> cd octave
  octave:7> which foo
  foo is the user-defined function from the file
  /scratch/jwe/build/octave/foo.m
  octave:8> foo
  ans = 1

In this example, there are two functions.  One in

  /scratch/jwe/build/octave/foo.m

that returns 1, and another in

  /scratch/jwe/build/foo.m

that returns 2.  I'm starting out in /scratch/jwe/build/octave.  It
seems that WHICH finds the correct one in each case, as does the
function lookup code.  The following also appears to me to be working
correctly:

  octave:1> foo
  ans = 1
  octave:2> cd ..
  octave:3> foo
  ans = 2
  octave:4> which foo
  foo is the user-defined function from the file
  /scratch/jwe/build/foo.m
  octave:5> cd octave
  octave:6> foo
  ans = 1
  octave:7> which foo
  foo is the user-defined function from the file
  /scratch/jwe/build/octave/foo.m

(evaluating before calling WHICH).  At least I think this is the right
behavior.  If not, please explain what is going wrong here, as I don't
see it.

| On another point what about the issue raised in
|
| http://www.octave.org/mailing-lists/octave-maintainers/2004/701
|
| for inline functions. Should octave be saving the curr_sym_tab, fbi_sym_tab,
| loadpath as context for the function handles? Can I implement copy functions
| for symbol_table, which would be needed for this? Or is there a reason why
| such a copy function should be avoided?

I suspect that copying the symbol tables is not actually necessary,
but I haven't had time to look at the problem in detail yet.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

David Bateman-3
In reply to this post by Teemu Ikonen

> Can you show precisely what the problem is?  Here is what I see:

Err my fault... My second test function was an empty file... You're
right it works with which as well.... Which means that a script file
(not a function) in the current scope also takes precedence. That is,
if one of your foo.m contains

function y = foo ()
 y = 1;
endfunction

and the other contains

1;
disp("foo");

then you get

octave:1> foo
ans = 1
octave:2> cd ..
octave:3> foo
foo
octave:4> which foo
foo is the user-defined function from the file
/home/dbateman/tmp/foo.m
octave:5> cd tmp
octave:6> which foo
foo is the user-defined function from the file
/home/dbateman/tmp/foo.m

I'd expect the first which to say

which: `foo' is the script file
/home/dbateman/foo.m

D.



--
David Bateman                                [hidden email]
Motorola CRM                                 +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax)
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as:

[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

John W. Eaton-6
In reply to this post by David Bateman-3
On 16-Sep-2004, David Bateman <[hidden email]> wrote:

| > Also, I'm wondering about the difference between inline functions and user
| > defined functions defined on the command line with the function keyword.
|
| An inline function deosn't appear in the fbi_sym_tab. Apart from that
| not much different. A function handle however has a subtle difference,
| for example
|
| octave:1> fcn = @not
| octave:2> function y = not (x), y = x; endfunction
| octave:3> fcn(1), not(1)
| ans = 0
| ans = 1
|
| This happens due to the fact that the function handle saves the
| octave_function in the original symbol table before it was replaced..
| However, this is not true for inline functions, for example
|
| octave:1> fcn = inline('not(x)')
| fcn =
|
| f(x) = not(x)
|
| octave:2> not = inline('x')
| not =
|
| f(x) = x
|
| octave:3> fcn(1), not(1)
| ans = 0
| ans = 0
|
| Err, in fact this seems to be a compatiability problem with matlab,
| as it seems matlab save a copy of the current symbol table for inline
| functions, since in matlab the above example gives
|
| >> fcn = inline('not(x)')
|
| fcn =
|
|      Inline function:
|      fcn(x) = not(x)
|
| >> not = inline('x')
|
| not =
|
|      Inline function:
|      not(x) = x
|
| >> fcn(1), not(1)
|
| ans =
|
|      0
|
|
| ans =
|
|      1

Please try the following patch.

I've checked this in, but unfortunately, the anonymous CVS archive is
down at the moment and probably won't be back until sometime tomorrow
(after around 9AM USA central time).

jwe


src/ChangeLog:

2004-09-16  John W. Eaton  <[hidden email]>

        * parse.y (frob_function): Clear id_name from curr_sym_tab, not
        top_level_sym_tab.

 
Index: src/parse.y
===================================================================
RCS file: /usr/local/cvsroot/octave/src/parse.y,v
retrieving revision 1.222
diff -u -r1.222 parse.y
--- a/src/parse.y 6 Sep 2004 20:19:57 -0000 1.222
+++ b/src/parse.y 17 Sep 2004 02:51:23 -0000
@@ -2715,7 +2715,21 @@
 
   fcn->stash_function_name (id_name);
 
-  top_level_sym_tab->clear (id_name);
+  // Enter the new function in fbi_sym_tab.  If there is already a
+  // variable of the same name in the current symbol table, we won't
+  // find the new function when we try to call it, so we need to clear
+  // the old symbol from the current symbol table.  Note that this
+  // means that for things like
+  //
+  //   function f () eval ("function g () 1, end"); end
+  //   g = 13;
+  //   f ();
+  //   g
+  //
+  // G will still refer to the variable G (with value 13) rather
+  // than the function G, until the variable G is cleared.
+
+  curr_sym_tab->clear (id_name);
 
   symbol_record *sr = fbi_sym_tab->lookup (id_name, true);
 


Reply | Threaded
Open this post in threaded view
|

Re: Crash with inline

David Bateman-3
According to John W. Eaton <[hidden email]> (on 09/17/04):
> Please try the following patch.
>
> I've checked this in, but unfortunately, the anonymous CVS archive is
> down at the moment and probably won't be back until sometime tomorrow
> (after around 9AM USA central time).
>

Ok, this seems to work..

D.

--
David Bateman                                [hidden email]
Motorola CRM                                 +33 1 69 35 48 04 (Ph)
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax)
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as:

[x] General Business Information
[ ] Motorola Internal Use Only
[ ] Motorola Confidential Proprietary