changes to path searching

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

changes to path searching

John W. Eaton-6
I've just checked in some changes that make Octave's path searching
behave more like Matlab.  Instead of searching for the entire
directory tree for .oct files, then searching again for .m files,
Octave now searches for .oct or .m files (in that order) in each
directory in the path.  This means that you can now create a .m file
that overrides a .oct file, provided that the .m file is located in a
directory that appears in the LOADPATH before the directory containing
the .oct file.  Previously, the .oct file would always be found
because Octave would search first for .oct files, then .m files.

You can also use file_in_loadpath to perform the same kind of search
with a command like

  file_in_loadpath ({"foo.oct", "foo.m"})

to search each directory in the loadpath for foo.oct then foo.m, and
return the first one found, or

  file_in_loadpath ({"foo.oct", "foo.m"}, "all")

if you want to find the locations of all the files, in the order
they are found.

The down side of this change is that it required changes to the
internals of the kpathsearch library.  I've submitted a patch to the
kpathsearch maintainers, but I don't know whether they will accept it.
So it may be a bit longer until we can use a "standard" version of
kpathsearch with Octave.

Comments?

jwe


Reply | Threaded
Open this post in threaded view
|

Re: changes to path searching

Joseph P. Skudlarek
> Octave now searches for .oct or .m files (in that order) in each
> directory in the path.
> ...
> Comments?

Sounds like a good choice -- improved functionality (can now override an
.oct file with an .m file), improved compatability (with MATLAB, and
with the way search works for PATH too) are a plus; potential backward
incompatability and delaying a standard version of kpathsearch seem like
small costs/risks, and worth taking.  

Thanks for creating and continuing to move Octave forward, John.  /Jskud


Reply | Threaded
Open this post in threaded view
|

Re: changes to path searching

Paul Kienzle-2
In reply to this post by John W. Eaton-6
On Thu, Dec 26, 2002 at 11:35:45PM -0600, John W. Eaton wrote:
> I've just checked in some changes that make Octave's path searching
> behave more like Matlab.  Instead of searching for the entire
> directory tree for .oct files, then searching again for .m files,
> Octave now searches for .oct or .m files (in that order) in each
> directory in the path.  This means that you can now create a .m file
> that overrides a .oct file, provided that the .m file is located in a
> directory that appears in the LOADPATH before the directory containing
> the .oct file.  Previously, the .oct file would always be found
> because Octave would search first for .oct files, then .m files.

What happens if for example you have logm and sqrtm in the same
oct-file on the system path and you put an m-file logm before it
on the path?  Do you still search through the loaded oct-files
before checking the path?  If that's the case, then you will get
different behaviour depending on whether you call sqrtm before or
after logm.  

I guess this is not a new problem --- if for example you have
an oct-file version of sqrtm on your octave-forge path, then you
better be sure to have an oct-file version of logm beside it so
that you don't get different behaviour depending on whether you
call sqrtm before or after logm.

Paul Kienzle
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: changes to path searching

John W. Eaton-6
On 27-Dec-2002, Paul Kienzle <[hidden email]> wrote:

| What happens if for example you have logm and sqrtm in the same
| oct-file on the system path and you put an m-file logm before it
| on the path?  Do you still search through the loaded oct-files
| before checking the path?

The current search is performed like this (load_fcn_from_file in
parse.y):

  ...

  static string_vector names (2);

  names[0] = nm + ".oct";
  names[1] = nm + ".m";

  std::string file
   = octave_env::make_absolute (Vload_path_dir_path.find_first_of (names),
                                octave_env::getcwd ());

  int len = file.length ();

  if (file.substr (len-4, len-1) == ".oct")
    {
      // Load the .oct file.  Previous versions of Octave only pass nm
      // (the function name) to the loader.  Now we also pass the
      // filename where we expect to find the symbol.

      if (octave_dynamic_loader::load (nm, file))
        force_link_to_function (nm);
    }
  else
    {
      // ... load the M-file.


In octave_dynamic_loader::load, we search the currently loaded .oct
files for the symbol before actually loading the file, so yes, I think
this is still a problem.  I suppose that instead of searching all the
currently loaded .oct files, we should only search the one that we are
currently loading (if it has already been loaded) otherwise, load it
and search.  This requires maintaining a list of filename to dynamic
library handles in the dynamic loader module, but that is easy enough
to do.  Would that solve the problem?

jwe


Reply | Threaded
Open this post in threaded view
|

Re: changes to path searching

Paul Kienzle-2
On Fri, Dec 27, 2002 at 02:52:29PM -0600, John W. Eaton wrote:
> In octave_dynamic_loader::load, we search the currently loaded .oct
> files for the symbol before actually loading the file, so yes, I think
> this is still a problem.  I suppose that instead of searching all the
> currently loaded .oct files, we should only search the one that we are
> currently loading (if it has already been loaded) otherwise, load it
> and search.  This requires maintaining a list of filename to dynamic
> library handles in the dynamic loader module, but that is easy enough
> to do.  Would that solve the problem?

That will only solve the problem if you can recognize that two files are
the same.  That should be easy enough, no?  If it is a symbolic link,
dereference it until you get to the base file, then check if that file is
already loaded, otherwise load it.  As a bonus, you don't have to search
through all loaded files.

This may make life slightly more difficult for some if they are relying on
the fact that for example myconstructor() will automatically load all
associated functions for their type and they don't need symlinks to
myconstructor.oct.

This will also make life difficult for a native windows port in that
windows has dodgy support for symbolic links.  I suppose I can write some
special code for windows so that xxx.lnk is a text file which contains the
relative path to the oct which exports the function.  I'll worry about that
when I have more time to work on the native port.

Paul Kienzle
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: changes to path searching

John W. Eaton-6
On 27-Dec-2002, Paul Kienzle <[hidden email]> wrote:

| On Fri, Dec 27, 2002 at 02:52:29PM -0600, John W. Eaton wrote:
| > In octave_dynamic_loader::load, we search the currently loaded .oct
| > files for the symbol before actually loading the file, so yes, I think
| > this is still a problem.  I suppose that instead of searching all the
| > currently loaded .oct files, we should only search the one that we are
| > currently loading (if it has already been loaded) otherwise, load it
| > and search.  This requires maintaining a list of filename to dynamic
| > library handles in the dynamic loader module, but that is easy enough
| > to do.  Would that solve the problem?
|
| That will only solve the problem if you can recognize that two files are
| the same.  That should be easy enough, no?  If it is a symbolic link,
| dereference it until you get to the base file, then check if that file is
| already loaded, otherwise load it.  As a bonus, you don't have to search
| through all loaded files.
|
| This may make life slightly more difficult for some if they are relying on
| the fact that for example myconstructor() will automatically load all
| associated functions for their type and they don't need symlinks to
| myconstructor.oct.
|
| This will also make life difficult for a native windows port in that
| windows has dodgy support for symbolic links.  I suppose I can write some
| special code for windows so that xxx.lnk is a text file which contains the
| relative path to the oct which exports the function.  I'll worry about that
| when I have more time to work on the native port.

OK, I'm convinced that what we are doing now is not the right thing,
and my simple idea of a filename to shared library handle map also has
some problems.

So what should happen when we need to load functions from .oct files?
If running a function from a single file (say "foo" from "foo.oct")
actually installs multiple functions, then I think we can come up with
a simple solution that will work OK, at least until someone clears
some functions, updates a .oct file, or installs a new .oct file in the
path ahead of the old one (forcing a reload).  Then I'm not sure what
should happen.  Currently reloading forces all functions that have
been loaded from a single file to be cleared.  But that will only work
if Octave was the one loading the functions into the symbol table --
if you do it with some code in your own loadable file, Octave won't
know about those functions, so it won't clear them.  Then I think
closing the shared library and reloading could cause some trouble.

jwe