GSoC Enquiry: Improving binary packaging

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

GSoC Enquiry: Improving binary packaging

Anirudha Bose
Hi.
First of all thanks a lot for selecting me as a GSoC student. It was a real surprise for me. Looking forward to a very productive summer ahead! :)

I was going through the list of accepted projects under GNU Project [1] when I saw that my mentor is Michael Goffioul. However the ideas page of Octave said that the mentor for this project will be either John W. Eaton or Jordi Gutiérrez Hermoso. Please clarify this and let me know.

My exams will be getting over tomorrow and I hope to start my work as soon as possible. Meanwhile I will submit all the documents required by Google and set up a blog to post my project updates.


Regards,
Anirudha Bose
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Jordi Gutiérrez Hermoso-2
Hi, welcome.

On 5 June 2013 00:27, Anirudha Bose <[hidden email]> wrote:
> I was going through the list of accepted projects under GNU Project [1] when
> I saw that my mentor is Michael Goffioul. However the ideas page of Octave
> said that the mentor for this project will be either John W. Eaton or Jordi
> Gutiérrez Hermoso. Please clarify this and let me know.

Michael is your mentor. I am already busy with two other projects.

- Jordi G. H.
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Michael Goffioul
In reply to this post by Anirudha Bose
On Wed, Jun 5, 2013 at 12:27 AM, Anirudha Bose <[hidden email]> wrote:
Hi.
First of all thanks a lot for selecting me as a GSoC student. It was a real surprise for me. Looking forward to a very productive summer ahead! :)

I was going through the list of accepted projects under GNU Project [1] when I saw that my mentor is Michael Goffioul. However the ideas page of Octave said that the mentor for this project will be either John W. Eaton or Jordi Gutiérrez Hermoso. Please clarify this and let me know.

Hi Anirudha,

I will be your mentor for this project. John and Jordi are already busy mentoring other projects, so I offered my services for this one. You might not know me, so let me briefly introduce myself. I've been part of the octave project for a couple of years (since 2007 iirc) and have been on and off, depending on the workload of my regular job. I worked on various things like: Win32 native port of octave, java integration, java-based graphics toolkit (JHandles), builtin octave graphics system, OpenGL renderer, text renderer, and more recently classdef implementation. I'm also maintaining MSVC-based binaries for Windows.

I'm an electronic engineer, but nowadays I'm mainly working in the software engineering domain.

My timezone is UTC-4, so I'm not sure we'll be able to communicate through IRC. Nevertheless, I advise you to hang out on IRC, as there are a bunch of people that will be able to help you if I'm not around. My IRC nickname is "goffioul".

One thing that I would like you to think about sooner rather than later is how you will integrate your project with the current octave code base. It doesn't mean it has to be part of octave source tree (for instance jwe's MXE is a separate repository), but as some point we'll need a plan to integrate your work into the octave "project".

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

John W. Eaton
Administrator
On 06/05/2013 12:14 PM, Michael Goffioul wrote:

> On Wed, Jun 5, 2013 at 12:27 AM, Anirudha Bose <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi.
>     First of all thanks a lot for selecting me as a GSoC student. It was
>     a real surprise for me. Looking forward to a very productive summer
>     ahead! :)
>
>     I was going through the list of accepted projects under GNU Project
>     [1] when I saw that my mentor is Michael Goffioul. However the ideas
>     page of Octave said that the mentor for this project will be either
>     John W. Eaton or Jordi Gutiérrez Hermoso. Please clarify this and
>     let me know.
>
>
> Hi Anirudha,
>
> I will be your mentor for this project. John and Jordi are already busy
> mentoring other projects, so I offered my services for this one. You
> might not know me, so let me briefly introduce myself. I've been part of
> the octave project for a couple of years (since 2007 iirc) and have been
> on and off, depending on the workload of my regular job. I worked on
> various things like: Win32 native port of octave, java integration,
> java-based graphics toolkit (JHandles), builtin octave graphics system,
> OpenGL renderer, text renderer, and more recently classdef
> implementation. I'm also maintaining MSVC-based binaries for Windows.
>
> I'm an electronic engineer, but nowadays I'm mainly working in the
> software engineering domain.
>
> My timezone is UTC-4, so I'm not sure we'll be able to communicate
> through IRC. Nevertheless, I advise you to hang out on IRC, as there are
> a bunch of people that will be able to help you if I'm not around. My
> IRC nickname is "goffioul".
>
> One thing that I would like you to think about sooner rather than later
> is how you will integrate your project with the current octave code
> base. It doesn't mean it has to be part of octave source tree (for
> instance jwe's MXE is a separate repository), but as some point we'll
> need a plan to integrate your work into the octave "project".

What are the plans for this project?  I'd like to see us converge on one
build system rather than duplicating effort (again).

I know I am guilty of duplicating previous work when I was trying to get
Octave to build with a MinGW cross compiler because I didn't really look
at what Michael and others had done in the past for building Octave on
Windows, but I'd like to see us avoid further duplication of effort.

jwe
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Michael Goffioul
On Thu, Jun 6, 2013 at 2:23 PM, John W. Eaton <[hidden email]> wrote:
On 06/05/2013 12:14 PM, Michael Goffioul wrote:
On Wed, Jun 5, 2013 at 12:27 AM, Anirudha Bose <[hidden email]
<mailto:[hidden email]>> wrote:

    Hi.
    First of all thanks a lot for selecting me as a GSoC student. It was
    a real surprise for me. Looking forward to a very productive summer
    ahead! :)

    I was going through the list of accepted projects under GNU Project
    [1] when I saw that my mentor is Michael Goffioul. However the ideas
    page of Octave said that the mentor for this project will be either
    John W. Eaton or Jordi Gutiérrez Hermoso. Please clarify this and
    let me know.


Hi Anirudha,

I will be your mentor for this project. John and Jordi are already busy
mentoring other projects, so I offered my services for this one. You
might not know me, so let me briefly introduce myself. I've been part of
the octave project for a couple of years (since 2007 iirc) and have been
on and off, depending on the workload of my regular job. I worked on
various things like: Win32 native port of octave, java integration,
java-based graphics toolkit (JHandles), builtin octave graphics system,
OpenGL renderer, text renderer, and more recently classdef
implementation. I'm also maintaining MSVC-based binaries for Windows.

I'm an electronic engineer, but nowadays I'm mainly working in the
software engineering domain.

My timezone is UTC-4, so I'm not sure we'll be able to communicate
through IRC. Nevertheless, I advise you to hang out on IRC, as there are
a bunch of people that will be able to help you if I'm not around. My
IRC nickname is "goffioul".

One thing that I would like you to think about sooner rather than later
is how you will integrate your project with the current octave code
base. It doesn't mean it has to be part of octave source tree (for
instance jwe's MXE is a separate repository), but as some point we'll
need a plan to integrate your work into the octave "project".

What are the plans for this project?  I'd like to see us converge on one build system rather than duplicating effort (again).

I know I am guilty of duplicating previous work when I was trying to get Octave to build with a MinGW cross compiler because I didn't really look at what Michael and others had done in the past for building Octave on Windows, but I'd like to see us avoid further duplication of effort.

I agree that there should be a single build system for "exotic" platforms. And I see this project as a way to merge the existing efforts and converge towards a single system. The project description is on google-melange, but basically there are 2 items on the table: MXE and GUB. The goal is to select one and make it produce binaries for as many platforms as possible.

For MXE, what you've done so far is a good starting point. I understand it's able to cross-compile to Windows from Linux, but also native-compile under Linux and MinGW(?). Anirudha, in your project description, you're stating that MXE cannot be used for OS X. Why is it so? Couldn't jwe's version of MXE be re-used to cross-/native-compile octave?

Concerning GUB, I don't know it, so I can't tell. But if jwe's MXE can be used, that would be my first choice, in order to leverage all the work that's already been done.

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Anirudha Bose
On Sat, Jun 8, 2013 at 4:19 AM, Michael Goffioul <[hidden email]> wrote:
On Thu, Jun 6, 2013 at 2:23 PM, John W. Eaton <[hidden email]> wrote:
What are the plans for this project?  I'd like to see us converge on one build system rather than duplicating effort (again). 
 
John, I read your update on cross-compiling update for MinGW systems [1], and I think it is better to improve your work rather than duplicating, like you said.

I followed the instructions in your post but I am stuck with an error while trying to build texinfo. Here is the log: http://pastebin.com/yaUwD9my

I know I am guilty of duplicating previous work when I was trying to get Octave to build with a MinGW cross compiler because I didn't really look at what Michael and others had done in the past for building Octave on Windows, but I'd like to see us avoid further duplication of effort.

I would like to know what other efforts had been done before for building Octave on Windows.

I agree that there should be a single build system for "exotic" platforms. And I see this project as a way to merge the existing efforts and converge towards a single system. The project description is on google-melange, but basically there are 2 items on the table: MXE and GUB. The goal is to select one and make it produce binaries for as many platforms as possible.

For MXE, what you've done so far is a good starting point. I understand it's able to cross-compile to Windows from Linux, but also native-compile under Linux and MinGW(?). Anirudha, in your project description, you're stating that MXE cannot be used for OS X. Why is it so? Couldn't jwe's version of MXE be re-used to cross-/native-compile octave?

Jordi had once told me about this. See this discussion [2].

[2] http://octave.1599824.n4.nabble.com/GSoC-2013-td4652333.html

Concerning GUB, I don't know it, so I can't tell. But if jwe's MXE can be used, that would be my first choice, in order to leverage all the work that's already been done.
 
Since a lot of work has already been done in MXE, I think I should go for it rather than starting from scratch using GUB.

--
Anirudha
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

John W. Eaton
Administrator
On 06/08/2013 09:00 AM, Anirudha Bose wrote:

> John, I read your update on cross-compiling update for MinGW systems
> [1], and I think it is better to improve your work rather than
> duplicating, like you said.
>
> I followed the instructions in your post but I am stuck with an error
> while trying to build texinfo. Here is the log: http://pastebin.com/yaUwD9my

I don't know what's causing that problem.  I'm not seeing it on my system.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

John Donoghue-3
In reply to this post by Anirudha Bose
Message: 1 Date: Sat, 08 Jun 2013 10:31:00 -0400 From: "John W. Eaton"
<[hidden email]> To: Anirudha Bose <[hidden email]> Cc: Octave
Maintainers List <[hidden email]> Subject: Re: GSoC
Enquiry: Improving binary packaging Message-ID:
<[hidden email]> Content-Type: text/plain; charset=UTF-8;
format=flowed On 06/08/2013 09:00 AM, Anirudha Bose wrote:

>> John, I read your update on cross-compiling update for MinGW systems
>> [1], and I think it is better to improve your work rather than
>> duplicating, like you said.
>>
>> I followed the instructions in your post but I am stuck with an error
>> while trying to build texinfo. Here is the log: http://pastebin.com/yaUwD9my
> I don't know what's causing that problem.  I'm not seeing it on my system.
>
> jwe
>
>
>

Not sure it is the actual cause of the problem, but it looks like it is
not finding the cross compiler:
checking for i686-pc-mingw32-strip... no
checking for strip... strip
configure: WARNING: using cross tools not prefixed with host triplet
checking whether to enable maintainer-specific portions of Makefiles... no
checking for i686-pc-mingw32-gcc... no
checking for gcc... gcc


Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Michael Goffioul
In reply to this post by Anirudha Bose
On Sat, Jun 8, 2013 at 9:00 AM, Anirudha Bose <[hidden email]> wrote:

I know I am guilty of duplicating previous work when I was trying to get Octave to build with a MinGW cross compiler because I didn't really look at what Michael and others had done in the past for building Octave on Windows, but I'd like to see us avoid further duplication of effort.

I would like to know what other efforts had been done before for building Octave on Windows.

There are (have been) a couple of them, I'll try to summarize as best as I can. In chronological order, we have:

1) MSVC-based compilation: over the years, I built a system to build octave and its dependencies based on shell scripts. The system consists mainly of a set of patches, tools and a few shell scripts to glue everything together. Initially, I had developed the system to be used for MSVC and MinGW, but the MinGW part never happened. It is available here [1], and a VS2010 precompiled version of all deps can be found here [2]. Note that [1] is not really up-to-date, I'll try to post an update when I get some time. One interesting that could be re-use from it is the installer building part.

2) MinGW-based compilation: this was initially done by Benjamin Lindner, who re-used part of my patches for MSVC (there's a lot of common things to fix for MinGW and MSVC). It's also based on shell script and can be found in octave-forge SVN [3]. I see that this link also contains stuff for cygwin and mingw64, but I don't know how up-to-date they're are and maintained.

3) MinGW-based compilation: as a continuation of Benjamin Lindner work, Tatsuro and (later) Nitzan started to produce binaries compiled with MinGW. Nitzan binaries are the one you can find in octave-forge "Files" section [4]. I'm not aware of the availability of the scripts/system he's using, but I might be wrong.

4) MXE/MinGW: recently John started to work on adapting MXE for our own use in octave. The result is available as a separate repo [5]. Initially started as a way to cross-compile octave for Windows from Linux, AFAIK it is able to be used for native compilation. I know that John has re-used some patch(es) from my system (at least for readline), but I don't know exactly how much has re-used.

5) Concerning OS X, I don't have much information to give. I'm not a OS X user and I didn't really follow the history of the corresponding binaries. I know that Ben Abbott used to build and provides binaries, but I don't know what's the current status. You can find some information on the wiki [6].

From this, I think a realistic target could be to start from jwe's MXE and make it able to compile octave (and deps) for:
- MinGW, cross-compile
- MinGW, native
- GCC/Linux, native (IIRC the target there was RHEL, which provides outdated/unsupported versions of critical required components)
- OS X, native (using what compiler? Xcode? GCC?)
- OS X, cross-compile (?): here, I'm not even sure this is possible

Michael.


Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Jordi Gutiérrez Hermoso-2
On 8 June 2013 14:17, Michael Goffioul <[hidden email]> wrote:
> From this, I think a realistic target could be to start from jwe's MXE and
> make it able to compile octave (and deps) for:
[snip]
> - OS X, native (using what compiler? Xcode? GCC?)
> - OS X, cross-compile (?): here, I'm not even sure this is possible

I don't believe it's possible to compile at all for Mac OS X without
clicking an "I Agree" button written by Apple or its moral equivalent.
So perhaps the best that can be done is using MXE for a native build
on Mac OS X, but this will require installing Xcode or its
stripped-down version. This means that anyone who wants to compile for
Mac OS X needs to agree being spied upon by Apple and agree to Apple's
NDA legalese, unless this legalese has changed since last time I saw
it.

A program that can automate building app bundles or .dmg files for Mac
OS X should be part of this project.

- Jordi G. H.
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

bpabbott
Administrator

On Jun 10, 2013, at 10:17 PM, Jordi Gutiérrez Hermoso wrote:

> On 8 June 2013 14:17, Michael Goffioul <[hidden email]> wrote:
>> From this, I think a realistic target could be to start from jwe's MXE and
>> make it able to compile octave (and deps) for:
> [snip]
>> - OS X, native (using what compiler? Xcode? GCC?)
>> - OS X, cross-compile (?): here, I'm not even sure this is possible
>
> I don't believe it's possible to compile at all for Mac OS X without
> clicking an "I Agree" button written by Apple or its moral equivalent.
> So perhaps the best that can be done is using MXE for a native build
> on Mac OS X, but this will require installing Xcode or its
> stripped-down version. This means that anyone who wants to compile for
> Mac OS X needs to agree being spied upon by Apple and agree to Apple's
> NDA legalese, unless this legalese has changed since last time I saw
> it.
>
> A program that can automate building app bundles or .dmg files for Mac
> OS X should be part of this project.
>
> - Jordi G. H.

Jordi,

Apparently, I'm not keeping up ... What benefit is MXE on MacOS X if Xcode is needed?

Ben
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Lukas Reichlin-4
In reply to this post by Michael Goffioul
On 08.06.2013, at 20:17, Michael Goffioul <[hidden email]> wrote:

> 5) Concerning OS X, I don't have much information to give. I'm not a OS X user and I didn't really follow the history of the corresponding binaries. I know that Ben Abbott used to build and provides binaries, but I don't know what's the current status. You can find some information on the wiki [6].

Regarding a real Octave.app for OS X, there was a "framework" [1] by Thomas Treichl for Octave 3.2 using Apple's Xcode.  This seems to be a superior way
to MacPorts-related binaries.  How about resurrecting Thomas' work?

Lukas

[1]
http://sourceforge.net/p/octave/code/HEAD/tree/trunk/octave-forge/admin/MacOSX/
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Jordi Gutiérrez Hermoso-2
In reply to this post by bpabbott
On 10 June 2013 10:23, Ben Abbott <[hidden email]> wrote:

> Apparently, I'm not keeping up ... What benefit is MXE on MacOS X if Xcode is needed?

It would be a single location for maintaining all build scripts. It
would pull in all dependencies except the build tools, and it would
work the same way across any OS. We would be able to maintain a single
build system for Octave for any OS.

It would be nice to be able to cross-compile (and I think it's
possible, if you click on "I Agree" and get the "Xcode developer
tools" package, which you can then use to bootstrap a cross-compiling
gcc), but even without that, the logic you're building by hand into
making a .dmg or app bundle can be folded into MXE.

- Jordi G. H.
Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Michael Goffioul
In reply to this post by Lukas Reichlin-4
On Mon, Jun 10, 2013 at 10:27 AM, Lukas Reichlin <[hidden email]> wrote:
On 08.06.2013, at 20:17, Michael Goffioul <[hidden email]> wrote:

> 5) Concerning OS X, I don't have much information to give. I'm not a OS X user and I didn't really follow the history of the corresponding binaries. I know that Ben Abbott used to build and provides binaries, but I don't know what's the current status. You can find some information on the wiki [6].

Regarding a real Octave.app for OS X, there was a "framework" [1] by Thomas Treichl for Octave 3.2 using Apple's Xcode.  This seems to be a superior way
to MacPorts-related binaries.  How about resurrecting Thomas' work?

Ideally, we should have a single build system. MXE seems to be good candidate; note that jwe's MXE does not mean "cross-compiling to MinGW" anymore, it's more than that: it's a Makefile-based build system that is able to do cross and native build under Linux and MinGW (wip). I have 0 knowledge of the OS X build systems, but is it imaginable to integrate existing work into MXE?

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: GSoC Enquiry: Improving binary packaging

Michael Goffioul
In reply to this post by bpabbott
On Mon, Jun 10, 2013 at 10:23 AM, Ben Abbott <[hidden email]> wrote:

On Jun 10, 2013, at 10:17 PM, Jordi Gutiérrez Hermoso wrote:

> On 8 June 2013 14:17, Michael Goffioul <[hidden email]> wrote:
>> From this, I think a realistic target could be to start from jwe's MXE and
>> make it able to compile octave (and deps) for:
> [snip]
>> - OS X, native (using what compiler? Xcode? GCC?)
>> - OS X, cross-compile (?): here, I'm not even sure this is possible
>
> I don't believe it's possible to compile at all for Mac OS X without
> clicking an "I Agree" button written by Apple or its moral equivalent.
> So perhaps the best that can be done is using MXE for a native build
> on Mac OS X, but this will require installing Xcode or its
> stripped-down version. This means that anyone who wants to compile for
> Mac OS X needs to agree being spied upon by Apple and agree to Apple's
> NDA legalese, unless this legalese has changed since last time I saw
> it.
>
> A program that can automate building app bundles or .dmg files for Mac
> OS X should be part of this project.
>
> - Jordi G. H.

Jordi,

Apparently, I'm not keeping up ... What benefit is MXE on MacOS X if Xcode is needed?

- single location for build script
- build automation, including deps (if required)
- automated installer generation

Michael.