package autoload

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

package autoload

Olaf Till-2
Let me first say that I'm sorry to speak up so late, now that pkg
autoload support recently has been removed. My excuse is that I think
such things should be discussed thoroughly at the mailing list -- not
only at OctConf or privately -- and I waited for this discussion,
thinking it would happen before such a change will be made. Maybe I
have just missed it, then it is wholly my fault.

There are two effects of this change:

1. A package can't force itself to be automatically loaded.

   Most OF packages already complied to the policy not to do this, and
   there was no considerable additional effort caused by this, so no
   real objection from me here.

2. The user has to change a startup file if he wants a package to be
   always loaded.

   I think this is awkward, and I'd like to explain why.

   Consider the case that a user installs a package with the intention
   to have it always loaded. Then he has to make changes at two
   different places: installing with 'pkg' and changing a startup
   file. Having to make changes at two different places for something
   which is intentionally an entity is not perfect design.

   If a user uninstalls a package which was always loaded, he has to
   remove a line from the startup file. Otherwise he'll get an error
   at next startup, when pkg tries to load the package though it isn't
   installed anymore.

   If one often installs and uninstalls packages (assuming one wants
   them to be always loaded if installed), the necessary changes in
   a startup file are a nuisance.

The things I mentioned as awkward could be avoided if there were a pkg
command to permanently note that a package should be always loaded: At
install, there'd be two different pkg commands necessary, or even only
one pkg command with an additional option, not changes in the
seemingly unrelated startup file. At uninstall, nothing additional
would be necessary at all, since pkg would automatically forget the
order to always load the package at package deinstallation.

So at install, instead of:

install package with pkg --- change startup file

you would have:

install package with pkg --- note package as autoload with pkg.

These are effectively the same schemes, only the latter is cleaner and
faster.

At uninstall, instead of:

uninstall package with pkg --- change startup file

you would just have:

uninstall package with pkg,

of which the latter "scheme" is clearly shorter.


Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

Juan Pablo Carbajal-2
On Fri, Apr 15, 2016 at 11:28 AM, Olaf Till <[hidden email]> wrote:

> Let me first say that I'm sorry to speak up so late, now that pkg
> autoload support recently has been removed. My excuse is that I think
> such things should be discussed thoroughly at the mailing list -- not
> only at OctConf or privately -- and I waited for this discussion,
> thinking it would happen before such a change will be made. Maybe I
> have just missed it, then it is wholly my fault.
>
> There are two effects of this change:
>
> 1. A package can't force itself to be automatically loaded.
>
>    Most OF packages already complied to the policy not to do this, and
>    there was no considerable additional effort caused by this, so no
>    real objection from me here.
>
> 2. The user has to change a startup file if he wants a package to be
>    always loaded.
>
>    I think this is awkward, and I'd like to explain why.
>
>    Consider the case that a user installs a package with the intention
>    to have it always loaded. Then he has to make changes at two
>    different places: installing with 'pkg' and changing a startup
>    file. Having to make changes at two different places for something
>    which is intentionally an entity is not perfect design.
>
>    If a user uninstalls a package which was always loaded, he has to
>    remove a line from the startup file. Otherwise he'll get an error
>    at next startup, when pkg tries to load the package though it isn't
>    installed anymore.
>
>    If one often installs and uninstalls packages (assuming one wants
>    them to be always loaded if installed), the necessary changes in
>    a startup file are a nuisance.
>
> The things I mentioned as awkward could be avoided if there were a pkg
> command to permanently note that a package should be always loaded: At
> install, there'd be two different pkg commands necessary, or even only
> one pkg command with an additional option, not changes in the
> seemingly unrelated startup file. At uninstall, nothing additional
> would be necessary at all, since pkg would automatically forget the
> order to always load the package at package deinstallation.
>
> So at install, instead of:
>
> install package with pkg --- change startup file
>
> you would have:
>
> install package with pkg --- note package as autoload with pkg.
>
> These are effectively the same schemes, only the latter is cleaner and
> faster.
>
> At uninstall, instead of:
>
> uninstall package with pkg --- change startup file
>
> you would just have:
>
> uninstall package with pkg,
>
> of which the latter "scheme" is clearly shorter.
>
>
> Olaf
>
> --
> public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

Olaf, wasn't you who recently argued in favor of following matlab
behavior regarding packages?
Toolboxes in matlab do not autoload (nor in any other language that I
know) without one modifying some startup file or including the toolbox
every time.
I might be wrong regarding the latest matlab versions. But if I am not
wrong, I must say that supporting opposing policies for similar Octave
behaviors is not helping us converge to a consensus.

Reply | Threaded
Open this post in threaded view
|

Re: package autoload

bpabbott
Administrator
> On Apr 15, 2016, at 07:47, Juan Pablo Carbajal <[hidden email]> wrote:
>
> Toolboxes in matlab do not autoload

I've been using Matlab since 1990. I don't use many toolboxes, but none of the toolboxes I have used require loading (optimization, for example).

Afaik, loading a toolbox is as simple as placing it in the path.

Ben

 
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

LachlanA
bpabbott wrote
> On Apr 15, 2016, at 07:47, Juan Pablo Carbajal <[hidden email]> wrote:
>
> Toolboxes in matlab do not autoload

I've been using Matlab since 1990. I don't use many toolboxes, but none of the toolboxes I have used require loading (optimization, for example).

Afaik, loading a toolbox is as simple as placing it in the path.

Ben
I think you are correct, Ben.  However, from what I gather, "loading" an Octave package is also simply adding it to the path.  Carnë explained on Tracker that we don't always want all the installed packages in the path because some conflict with each other, and some overload built-in functions.

My personal belief is that the __unimplemented__ function, which currently tells us which package we should load, should just load the package and continue (unless the same function is implemented in multiple packages, in which case it should list all of those packages and give an error).

That would most closely emulate the Matlab experience, while avoiding the problems Carnë described.

Cheers,
Lachlan
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

Oliver Heimlich
Am 15.04.2016 13:42 schrieb LachlanA:

> bpabbott wrote
>>> On Apr 15, 2016, at 07:47, Juan Pablo Carbajal &lt;
>
>> ajuanpi+dev@
>
>> &gt; wrote:
>>>
>>> Toolboxes in matlab do not autoload
>>
>> I've been using Matlab since 1990. I don't use many toolboxes, but
>> none of
>> the toolboxes I have used require loading (optimization, for example).
>>
>> Afaik, loading a toolbox is as simple as placing it in the path.
>>
>> Ben
>
> I think you are correct, Ben.  However, from what I gather, "loading"
> an
> Octave package is also simply adding it to the path.  Carnë explained
> on
> Tracker that we don't always want all the installed packages in the
> path
> because some conflict with each other, and some overload built-in
> functions.
>
> My personal belief is that the __unimplemented__ function, which
> currently
> tells us which package we should load, should just load the package and
> continue (unless the same function is implemented in multiple packages,
> in
> which case it should list all of those packages and give an error).
>
> That would most closely emulate the Matlab experience, while avoiding
> the
> problems Carnë described.
>
> Cheers,
> Lachlan


Removal of autoload will (hopefully) have one big advantage: That people
add “pkg load” commands at the beginning of their script files. This
will make the script files portable(*) and other users can easily tell
that they miss a particular package if the script stops right at the
start on their computer.

I find this particularly important in scientific publications when you
can easily tell from a script which packages the author has intended to
use. Compare this with other languages where you have
import-/use-statements for functions that your source code uses (and you
need in your namespace).

(*) although it might break compatibility with MATLAB, I don't know.

Oliver

Reply | Threaded
Open this post in threaded view
|

Re: package autoload

bpabbott
Administrator
In reply to this post by LachlanA
> On Apr 15, 2016, at 07:42, LachlanA <[hidden email]> wrote:
>
> bpabbott wrote
>>> On Apr 15, 2016, at 07:47, Juan Pablo Carbajal &lt;
>
>> ajuanpi+dev@
>
>> &gt; wrote:
>>>
>>> Toolboxes in matlab do not autoload
>>
>> I've been using Matlab since 1990. I don't use many toolboxes, but none of
>> the toolboxes I have used require loading (optimization, for example).
>>
>> Afaik, loading a toolbox is as simple as placing it in the path.
>>
>> Ben
>
> I think you are correct, Ben.  However, from what I gather, "loading" an
> Octave package is also simply adding it to the path.  Carnë explained on
> Tracker that we don't always want all the installed packages in the path
> because some conflict with each other, and some overload built-in functions.
>
> My personal belief is that the __unimplemented__ function, which currently
> tells us which package we should load, should just load the package and
> continue (unless the same function is implemented in multiple packages, in
> which case it should list all of those packages and give an error).
>
> That would most closely emulate the Matlab experience, while avoiding the
> problems Carnë described.

I often have trouble with m-file name conflicts between packages and/or core. Thus I prefer loading, and lean away from auto loading.

In any event, I only replied to clarify Matlab behavior.

Ben
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

Juan Pablo Carbajal-2
In reply to this post by bpabbott
On Fri, Apr 15, 2016 at 1:54 PM, Ben Abbott <[hidden email]> wrote:

>> On Apr 15, 2016, at 07:47, Juan Pablo Carbajal <[hidden email]> wrote:
>>
>> Toolboxes in matlab do not autoload
>
> I've been using Matlab since 1990. I don't use many toolboxes, but none of the toolboxes I have used require loading (optimization, for example).
>
> Afaik, loading a toolbox is as simple as placing it in the path.
>
> Ben
>
>
adding it to the path is modifying the starup file of matlab,
basically adding "addpath".
The user does it via the GUI, but they are modifying a configuration
file. You need toto this when you want a toolbox to be autoloaded, and
if you install it you need to remove it form there.
This is what the user needs to do with their .octaverc

Reply | Threaded
Open this post in threaded view
|

Re: package autoload

Olaf Till-2
In reply to this post by Oliver Heimlich
On Fri, Apr 15, 2016 at 02:55:47PM +0200, [hidden email] wrote:
> Removal of autoload will (hopefully) have one big advantage: That people add
> “pkg load” commands at the beginning of their script files. This will make
> the script files portable(*) and other users can easily tell that they miss
> a particular package if the script stops right at the start on their
> computer.

Without trying to abort the discussion, I'd like to say that this
convinces me to a sufficient degree.

And thanks to all who took the time to respond.

An afterthought (occured to me thinking about this all): I'd say at
least within OF, packages should usually try to avoid name conflicts
(except when deliberately overriding something). With a central list
of names, e.g.. It was only meant as a hint -- if you like to discuss
it, better start a new thread.

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

LachlanA
In reply to this post by Oliver Heimlich
Oliver Heimlich wrote
Removal of autoload will (hopefully) have one big advantage: That people
add “pkg load” commands at the beginning of their script files. This
will make the script files portable(*) and other users can easily tell
that they miss a particular package if the script stops right at the
start on their computer.

I find this particularly important in scientific publications when you
can easily tell from a script which packages the author has intended to
use. Compare this with other languages where you have
import-/use-statements for functions that your source code uses (and you
need in your namespace).

(*) although it might break compatibility with MATLAB, I don't know.

Oliver
Yes, it would break matlab compatibility -- unless included within "if exists ("OCTAVE_VERSION)" or some such.  I believe the nearest Matlab equivalent would be "import  packagename.*", though that may do something different.  That command also allows a subset of the functions to be imported, which is useful for avoiding the overloading of core function names.

Including the pkg command within the script also breaks scripts any time that the packages are renamed or functions are shuffled between them, though that doesn't happen all that often.
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

Olaf Till-2
On Fri, Apr 15, 2016 at 10:15:10PM -0700, LachlanA wrote:
> Yes, it would break matlab compatibility -- unless included within "if
> exists ("OCTAVE_VERSION)" or some such.

I'd think we mostly care for compatibility in the other
direction... Being able to run Matlab code in Octave is a declared
goal of Octave (Matlab replacement), but not the other way around. But
running Octave code in Matlab may be an issue for some, of course ...

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

PhilipNienhuis
In reply to this post by Oliver Heimlich
Oliver Heimlich wrote
<snip>

Removal of autoload will (hopefully) have one big advantage: That people
add “pkg load” commands at the beginning of their script files. This
will make the script files portable(*) and other users can easily tell
that they miss a particular package if the script stops right at the
start on their computer.
That would be great if at the end of that script file the package is unloaded as well.
I'm not fond of scripts that manipulate environments w/o cleaning up afterwards (unless such environment changes are intended for regular operation of SW).

Maybe a sort of "local"switch is required that loads packages only in the scope of functions and automatically unloads them at function exit.

I find this particularly important in scientific publications when you
can easily tell from a script which packages the author has intended to
use. Compare this with other languages where you have
import-/use-statements for functions that your source code uses (and you
need in your namespace).
Hmmm, looks a bit like a niche problem to me.
While you might see what packages are used you still can't see which versions.
IMO it's more a peer reviewer's job to check for completeness of code snippets.

Philip
Reply | Threaded
Open this post in threaded view
|

Re: package autoload

Oliver Heimlich
On 16.04.2016 19:43, PhilipNienhuis wrote:

> Oliver Heimlich wrote
>>
>> <snip>
>> Removal of autoload will (hopefully) have one big advantage: That people
>> add “pkg load” commands at the beginning of their script files. This
>> will make the script files portable(*) and other users can easily tell
>> that they miss a particular package if the script stops right at the
>> start on their computer.
>
> That would be great if at the end of that script file the package is
> unloaded as well.
> I'm not fond of scripts that manipulate environments w/o cleaning up
> afterwards (unless such environment changes are intended for regular
> operation of SW).
>
> Maybe a sort of "local"switch is required that loads packages only in the
> scope of functions and automatically unloads them at function exit.

Script files and function files are different. Also I find loading the
package more important than unloading (unless you have a package that
overrides and thereby breaks stuff).

Please note that the -local switch is already implemented but does
something different. Loading and unloading packages as part of a
function call has the potential to become a messy thing as well.

>> I find this particularly important in scientific publications when you
>> can easily tell from a script which packages the author has intended to
>> use. Compare this with other languages where you have
>> import-/use-statements for functions that your source code uses (and you
>> need in your namespace).
>
> Hmmm, looks a bit like a niche problem to me.
> While you might see what packages are used you still can't see which
> versions.
> IMO it's more a peer reviewer's job to check for completeness of code
> snippets.

The idea is that you can copy and run shared scripts easily. It does not
depend on a “pkg load” command in your startup files, which might have
been present at the developer's computer.

The version is not so much a problem: First, the author should have
cited the particular version used. Second, unless the package goes
through compatibility breaking changes you can still run the script with
a newer / the latest version of a package.

Oliver

P.S. Philip, your nameserver seems to be down. I can't answer you
directly. Hopefully you find this message in the mailing list.

Reply | Threaded
Open this post in threaded view
|

Re: package autoload

PhilipNienhuis
Oliver Heimlich wrote
On 16.04.2016 19:43, PhilipNienhuis wrote:
> Oliver Heimlich wrote
>>
>> <snip>
>> Removal of autoload will (hopefully) have one big advantage: That people
>> add “pkg load” commands at the beginning of their script files. This
>> will make the script files portable(*) and other users can easily tell
>> that they miss a particular package if the script stops right at the
>> start on their computer.
>
> That would be great if at the end of that script file the package is
> unloaded as well.
> I'm not fond of scripts that manipulate environments w/o cleaning up
> afterwards (unless such environment changes are intended for regular
> operation of SW).
>
> Maybe a sort of "local"switch is required that loads packages only in the
> scope of functions and automatically unloads them at function exit.

Script files and function files are different. Also I find loading the
package more important than unloading (unless you have a package that
overrides and thereby breaks stuff).
"breaks stuff" is exactly my "gripe".
E.g., shadowing functions (core, package or one's own functions) is an example.

I'm a fan of cleaning up after the work is done.  Like many other people I wasn't quite born that way, I had to learn the hard way later in life :-)

BTW, many many Matlab scripts I've seen start with "clear all; close all;".  For Octave that would imply to also include "pkg unload all".

Please note that the -local switch is already implemented but does
something different.
That's why I double-quoted it.
I should have been more specific, i.e. comparing it to the "local" switch in warnings
But I think the intention is clear.

Loading and unloading packages as part of a
function call has the potential to become a messy thing as well.
Unless it is the "outer"one where everything needed for some special application could be set up and initialized in the scope of that function and its descendants.
(and cleaned up afterwards, but that doesn't seem popular)

>> I find this particularly important in scientific publications when you
>> can easily tell from a script which packages the author has intended to
>> use. Compare this with other languages where you have
>> import-/use-statements for functions that your source code uses (and you
>> need in your namespace).
>
> Hmmm, looks a bit like a niche problem to me.
> While you might see what packages are used you still can't see which
> versions.
> IMO it's more a peer reviewer's job to check for completeness of code
> snippets.

The idea is that you can copy and run shared scripts easily. It does not
depend on a “pkg load” command in your startup files, which might have
been present at the developer's computer.

The version is not so much a problem: First, the author should have
cited the particular version used.
So then you also know what packages the author used, no?

Anyway, I do not disagree with your intention at all; for a scientific -and thus specialized- applications, and for other specialized situations like testing, your suggestion is helpful.
But bluntly advising to always add "pkg load" commands to every script file (that is the way I read your argument) can make for a messy situation later on. I'd be more careful with such advice.

P.S. Philip, your nameserver seems to be down. I can't answer you
directly. Hopefully you find this message in the mailing list.
I posted through nabble as I am not subscribed to any Octave mailing list. Heaven knows how nabble links my email address to thread postings, but usually it does a good job once I posted.

nabble does have a few quirks though (e.g., the irritating super-resume pop-up that even avoids uBlock-origin).

Thanks,

Philip