OF: community packages depending on external packages?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

OF: community packages depending on external packages?

Olaf Till-2
Hi,

in recent package release submissions, the issue came up whether
OF packages in the "community" category should be allowed to depend on
packages in the "external" community.

I think not. The "community" category was meant for a tight connection
with the development of core Octave. For this, our developer community
was meant to have control over the "community" packages. This control
is necessary, among others, for namespace issues.

"External" packages, on the other hand, are under the sole control of
their individual maintainers. Sometimes, they indirectly depend on
m-code developed only for Matlab.

If "community" packages depend on "external" packages, they depend on
m-code which is not under the cotrol of our developer community. I
think this would be a mistake.

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

PhilipNienhuis
Olaf Till-2 wrote

> Hi,
>
> in recent package release submissions, the issue came up whether
> OF packages in the "community" category should be allowed to depend on
> packages in the "external" community.
>
> I think not. The "community" category was meant for a tight connection
> with the development of core Octave. For this, our developer community
> was meant to have control over the "community" packages. This control
> is necessary, among others, for namespace issues.
>
> "External" packages, on the other hand, are under the sole control of
> their individual maintainers. Sometimes, they indirectly depend on
> m-code developed only for Matlab.
>
> If "community" packages depend on "external" packages, they depend on
> m-code which is not under the cotrol of our developer community. I
> think this would be a mistake.

Thanks for bringing this up on the maintainers ML, Olaf.

There was a previous discussion, initially confined to the package release
tracker for geometry-4.0.0 and its dependence on the matgeom package (in
turn depending on a matgeom toolbox). Now that gets a wider audience, which
IMO is good. I think it would also be good to bring up that discussion,
starting here:
https://sourceforge.net/p/octave/package-releases/376/#cba1

While theoretically you might be right, I think from a more practical POV
this interpretation may lead to less enthusiasm and maybe even frustration
for existing and potential OF package maintainers, including myself.

If we follow your reasoning the mapping package (depending on geometry and
matgeom, in the future also on octproj) must become external as well, maybe
even io as that also depends on external SW beyond our control (like Java
libraries for spreadheet I/O other than .xlsx, .ods and .gnumeric, some of
which are effectively abandoned now).
On the plus side, I could then stop nitpicking about Octave coding style for
contributed functions on the patch tracker, and such a change surely would
help speed up development.

Looking at some external OF packages with your reasoning in mind, e.g.,
octproj and octclip, I wonder why those are external in the first place.
Conversely, the (community) video package depends on a very specific
external library maintained by a third party (= not under our control).

As to namespaces, AFAIK the Matlab / Octave language doesn't have that
(yet?) so I suppose that point is moot.

Anyway, it is too bad that this subject, dependence on external packages,
wasn't touched in sufficient detail during the discussion about community
and external packages some years ago.

Philip




--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

Mike Miller-4
In reply to this post by Olaf Till-2
On Tue, Sep 03, 2019 at 11:42:23 +0200, Olaf Till wrote:

> Hi,
>
> in recent package release submissions, the issue came up whether
> OF packages in the "community" category should be allowed to depend on
> packages in the "external" community.
>
> I think not. The "community" category was meant for a tight connection
> with the development of core Octave. For this, our developer community
> was meant to have control over the "community" packages. This control
> is necessary, among others, for namespace issues.
I think yes, community packages should be allowed to depend on external
packages. Not unconditionally, but after considering the specifics of
the package and the dependency. I agree with the rest of this paragraph.

> "External" packages, on the other hand, are under the sole control of
> their individual maintainers. Sometimes, they indirectly depend on
> m-code developed only for Matlab.
>
> If "community" packages depend on "external" packages, they depend on
> m-code which is not under the cotrol of our developer community. I
> think this would be a mistake.

If a community package contains C++ code that links with an external
library, for example GSL or MPFR, then it also depends on code which is
not under the control of the Octave Forge developer community. The
miscellaneous community package contains a function that depends on the
GNU units external program. These scenarios seem similar to me and
should all be considered and reviewed critically with the possibility of
being accepted as community packages.

Can you explain why you think that depending on externally developed
m-file functions would be unacceptable for a community package, but
depending on externally developed C or C++ functions or programs would
be acceptable?

--
mike

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

apjanke-floss
In reply to this post by PhilipNienhuis

On 9/4/19 5:06 PM, PhilipNienhuis wrote:

> As to namespaces, AFAIK the Matlab / Octave language doesn't have that
> (yet?) so I suppose that point is moot.
>

Matlab/Octave totally have namespaces (though I think they're called
"packages" in the doco). You can see them in use in my Tablicious package.

Cheers,
Andrew

Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

Olaf Till-2
In reply to this post by Mike Miller-4
On Wed, Sep 04, 2019 at 04:06:01PM -0500, PhilipNienhuis wrote:
> If we follow your reasoning the mapping package (depending on geometry and
> matgeom, in the future also on octproj) must become external as well, maybe
> even io as that also depends on external SW beyond our control (like Java
> libraries for spreadheet I/O other than .xlsx, .ods and .gnumeric, some of
> which are effectively abandoned now).

I think external libraries (C, Java, or Python) are a different thing,
please see below.

On Wed, Sep 04, 2019 at 02:41:06PM -0700, Mike Miller wrote:
> Can you explain why you think that depending on externally developed
> m-file functions would be unacceptable for a community package, but
> depending on externally developed C or C++ functions or programs would
> be acceptable?

I can't explain it. It's rather a basic decision on whether we want
Octave and the "community" packages to be as a group self-contained
with respect to m-code (and "oct-code") or not.

It seems clear to me that a border must be set somewhere, which I'll
try to illustrate with the following examples.

Consider the following partially speculative case: Octave has
delegated many of its much used statistics functions into the
statistics package, probably under the premise that there they still
are under control of our community. What if we decided to let these
statistics functions depend on "upstream" code developed for Matlab?

And consider this corner (?) case, too: A formal "community" package
provides only glue to incorporate an "external" package, by
explicitely depending on it. Then the "community" package is de facto
"external".

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

PhilipNienhuis
In reply to this post by apjanke-floss
apjanke-floss wrote
> On 9/4/19 5:06 PM, PhilipNienhuis wrote:
>
>> As to namespaces, AFAIK the Matlab / Octave language doesn't have that
>> (yet?) so I suppose that point is moot.
>>
>
> Matlab/Octave totally have namespaces (though I think they're called
> "packages" in the doco). You can see them in use in my Tablicious package.

At first I thought you meant OF packages but Matlab does have a "package"
concept based on classdef. So yes there is a namespace, I stand corrected.

Philip



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

PhilipNienhuis
In reply to this post by Olaf Till-2
Olaf Till-2 wrote

> On Wed, Sep 04, 2019 at 04:06:01PM -0500, PhilipNienhuis wrote:
>> If we follow your reasoning the mapping package (depending on geometry
>> and
>> matgeom, in the future also on octproj) must become external as well,
>> maybe
>> even io as that also depends on external SW beyond our control (like Java
>> libraries for spreadheet I/O other than .xlsx, .ods and .gnumeric, some
>> of
>> which are effectively abandoned now).
>
> I think external libraries (C, Java, or Python) are a different thing,
> please see below.
>
> On Wed, Sep 04, 2019 at 02:41:06PM -0700, Mike Miller wrote:
>> Can you explain why you think that depending on externally developed
>> m-file functions would be unacceptable for a community package, but
>> depending on externally developed C or C++ functions or programs would
>> be acceptable?
>
> I can't explain it. It's rather a basic decision on whether we want
> Octave and the "community" packages to be as a group self-contained
> with respect to m-code (and "oct-code") or not.

Ah OK, "self-contained" is the central issue. Yes that would set external
C++/Java/Python/<whatever> code & libraries apart from Octave style .m and
.oct files referencing just other Octave style .m and .oct files.
But what are the benefits of that?

When the concept of community and external packages came up (at or around
OctConf 2018 IIRC) I had the impression that it was more about a limited
collection of well-maintained packages with often used functions, versus
much more specialized -and often less actively maintained if not orphaned-
packages.
(The current community packages tally is 49 so maybe I misunderstood
"limited".)
IIRC it was also about limiting the maintenance burden of core Octave by
keeping it lean and mean. Rather than absorb functions from OF packages into
core Octave, Octave would mainly comprise general functions while more
specialized functions would live in more specialized (community) packages.
Moving many statistics functions into the statistics package is a nice
example.


> It seems clear to me that a border must be set somewhere, which I'll
> try to illustrate with the following examples.
>
> Consider the following partially speculative case: Octave has
> delegated many of its much used statistics functions into the
> statistics package, probably under the premise that there they still
> are under control of our community. What if we decided to let these
> statistics functions depend on "upstream" code developed for Matlab?

As long as the upstream licenses are compatible with the GPL and the
upstream code is well-maintained, what would be the actual problem? After
all, Octave strives to be Matlab-compatible.

We could still reverse that decision, or change to other upstream code, as
we still control that statistics package's contents. IOW we are completely
free to decide about "our" dependencies.


> And consider this corner (?) case, too: A formal "community" package
> provides only glue to incorporate an "external" package, by
> explicitely depending on it. Then the "community" package is de facto
> "external".

Nothing would prevent the Octave community from glueing it to something
else.
I see no difference with depending on external C++/Python/Java/<whatever>
code.

In a more general sense, OF packages, whether community or external,
constitute an elegant way to link Octave to external code (hopefully of good
quality). To that end a "self-contained" concept seems unduly limiting to
me.


Philip



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

Juan Pablo Carbajal-2
Hi,

I am already feeling the frustration mentioned by Philip.
I want Octave to grow, to capture more user and connect with a fast
growing community of scientific programmers (python, Julia), and I am
more often than not put aback by this kind of discussions.
I want to make Octave attractive for the user, I believe we need to
oil the user-->dev pathway. I do not feel the vibe...
But, hey, that's alright.

What is the meaning of "code under control" in a libre software world?
It seems to me it is an unnatural border (aren't they all unnatural?)

As Philip well remembers, when we created the "categories" it was to
provide a "certificate of usefulness": compatibility with Octave core,
instalability, maintenance, documentation, etc... and license, yes,
yes
Nothing to do with "control", and much less with "code control".
In brief, a user installing a community package should expect no
troubles and be set for an excellent UX. With externals that cannot be
given for granted.

Just take the example of thriving community like pipy, and the julia
index. It is not about control, it is about well defined and
automatized ways of creating packages for adding functionality (the
metadata helps any potential reviewer...mainly users!)
Help the user check what they are getting (website, doc, code,
examples), do not play police!

So I vouch for completely abandoning any sense of "control over the
code", and start thinking about control over the UX of the package
(and eventually of the package manager...ahhh pkg I am looking at you
with disdain).

We are a community of volunteers, no foundation nor company backs us
up. We have to find slim&clever ways of achieving our goals or we will
be stuck in the mud.
And we have to set goals that our community (and man power) can
reach... that starts by reducing bureaucracy.

Cheers

Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

PhilipNienhuis
In reply to this post by Olaf Till-2
Olaf Till-2 wrote

> Hi,
>
> in recent package release submissions, the issue came up whether
> OF packages in the "community" category should be allowed to depend on
> packages in the "external" community.
>
> I think not. The "community" category was meant for a tight connection
> with the development of core Octave. For this, our developer community
> was meant to have control over the "community" packages. This control
> is necessary, among others, for namespace issues.
>
> "External" packages, on the other hand, are under the sole control of
> their individual maintainers. Sometimes, they indirectly depend on
> m-code developed only for Matlab.
>
> If "community" packages depend on "external" packages, they depend on
> m-code which is not under the cotrol of our developer community. I
> think this would be a mistake.

What will happen now? is this discussion dropped / ignored?

Given that a new mapping package release is actually ready for a long long
time now (> 2 years) and it depends on new geometry and matgeom releases I'd
like to know what to do next.

If mapping is to become an external package, together with geometry and
matgeom, fine with me, then users can at least benefit from a new release.
Too bad for the concept of community packages then.

The way this process has been lingering on doesn't appear nice to me.

Philip




--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: OF: community packages depending on external packages?

Olaf Till-2
On Wed, Oct 02, 2019 at 12:32:18PM -0500, PhilipNienhuis wrote:

> Olaf Till-2 wrote
> > Hi,
> >
> > in recent package release submissions, the issue came up whether
> > OF packages in the "community" category should be allowed to depend on
> > packages in the "external" community.
> >
> > I think not. The "community" category was meant for a tight connection
> > with the development of core Octave. For this, our developer community
> > was meant to have control over the "community" packages. This control
> > is necessary, among others, for namespace issues.
> >
> > "External" packages, on the other hand, are under the sole control of
> > their individual maintainers. Sometimes, they indirectly depend on
> > m-code developed only for Matlab.
> >
> > If "community" packages depend on "external" packages, they depend on
> > m-code which is not under the cotrol of our developer community. I
> > think this would be a mistake.
>
> What will happen now? is this discussion dropped / ignored?
>
> Given that a new mapping package release is actually ready for a long long
> time now (> 2 years) and it depends on new geometry and matgeom releases I'd
> like to know what to do next.
>
> If mapping is to become an external package, together with geometry and
> matgeom, fine with me, then users can at least benefit from a new release.
Since I was the only one expressing the opinion that community
packages should not depend on external packages I consider this
opinion to be overruled.

As for the external package matgeom, I indicated at the release forum
that I see an issue which prevents me from publishing the release. (I
didn't look further into this package as yet to see if this is the
only issue it has.)

As for the geometry package, I thought it was clear that it can't be
released as long as the matgeom package, on which it depends, is not
released. (I didn't look further into the geometry package as yet to
see if this is the only issue with it.)

Olaf

--
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc (849 bytes) Download Attachment