Fwd: Octave JIT consultancy

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: Octave JIT consultancy

Richard Crozier
Dear List,

I asked previously about the possibility of funding some development of
the Octave JIT (see below, I don't think I received any replies). I have
a company and might be in a position to do this in the future. I don't
think I received any reply, is there any interest in this at all?

Further to what I wrote below, there is also this:

https://developer.ibm.com/code/2016/07/19/jitbuilder-library-and-eclipse-omr-just-in-time-compilers-made-easy/

The source of this JITBuilder library is published on github under the
Eclipse OMR licence:

https://github.com/eclipse/omr/blob/master/LICENSE

This actually states though that you can relicense it under GPL v2 (or
some other licences).

More blog posts on it by the dev:

https://developer.ibm.com/code/author/mstoodle/

It seems to be under active development, the actual library source is here:

https://github.com/eclipse/omr/tree/master/jitbuilder

It uses the open source IMB Eclipse OMR compiler


Regards,

Richard


-------- Forwarded Message --------
Subject: Octave JIT
Date: Wed, 20 Mar 2019 09:45:01 +0000
From: Richard Crozier <[hidden email]>
To: [hidden email]

Dear List,

I wonder if the following new(ish) library might ease the process of
developing the Octave JIT:

https://github.com/jmmartinez/easy-just-in-time

There's a blog post here:

https://blog.quarkslab.com/easyjit-just-in-time-compilation-for-c.html

It seems to hide much of the complexity of LLVM (and the need to update
for the LLVM API, as the library maintainers will hopefully do this).

I am also wondering about the possibility of funding some work on the
JIT in the future. If any of the core devs is in a position to do this
it would be interesting to get an idea of what it might cost to make
some substantial progress on the JIT.

Regards,

Richard
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Octave JIT consultancy

Jordi Gutiérrez Hermoso-2
On Wed, 2019-05-08 at 08:37 +0000, CROZIER Richard wrote:
> I asked previously about the possibility of funding some development of
> the Octave JIT (see below, I don't think I received any replies).

So, I believe we have some money that could pay for this. I think it's
a good use of our money. This library also seems like a good approach.

Do you have someone in mind who could do the work?

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Octave JIT consultancy

Richard Crozier

On 11/06/2019 18:55, Jordi Gutiérrez Hermoso wrote:
> On Wed, 2019-05-08 at 08:37 +0000, CROZIER Richard wrote:
>> I asked previously about the possibility of funding some development of
>> the Octave JIT (see below, I don't think I received any replies).
>
> So, I believe we have some money that could pay for this. I think it's
> a good use of our money. This library also seems like a good approach.
>
> Do you have someone in mind who could do the work?
>

So, to be clear, are you suggesting you could also contribute money to a
project to work on this?

I don't have anyone in mind to do the work, I was hoping someone on the
list might be able to do it, or be able to suggest someone who could.

I  need to get an idea of the costs of getting work done. To be clear
again, we don't actually have a budget right now to fund it, but can
work it into future plans if we know it could be done and someone was
available to do it. I also know at least one other company who might be
interested in contributing, beyond my own.

Regards,

Richard
The University of Edinburgh is a charitable body, registered in Scotland, with the registration number SC005336.
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Octave JIT consultancy

Jordi Gutiérrez Hermoso-2
On Mon, 2019-06-17 at 11:57 +0000, CROZIER Richard wrote:

> On 11/06/2019 18:55, Jordi Gutiérrez Hermoso wrote:
> > On Wed, 2019-05-08 at 08:37 +0000, CROZIER Richard wrote:
> > > I asked previously about the possibility of funding some development of
> > > the Octave JIT (see below, I don't think I received any replies).
> > 
> > So, I believe we have some money that could pay for this. I think it's
> > a good use of our money. This library also seems like a good approach.
> > 
> > Do you have someone in mind who could do the work?
> > 

> So, to be clear, are you suggesting you could also contribute money to a
> project to work on this?

Yeah, Octave was given a 90k USD donation a while ago (10% of our
donation money goes to the FSF for processing fees):

https://www.fsf.org/news/free-software-foundation-receives-1-million-from-handshake

We have a bit more than that from other donations. As far as I'm
aware, we haven't used the money for anything yet. A competent
developer who could give us stable JIT compiling for Octave seems like
good use of this money, hopefully with some documentation so that we
can keep maintaining it after the work is done.

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Octave JIT consultancy

roland65
Perhaps you should directly contact the guys at the llvm or libgccjit
projects?

I recently tried this benchmark
https://github.com/jesusfv/Comparison-Programming-Languages-Economics in
Octave and the results were catastrophic due to the lack of a JIT
compiler...

This would be a major step forward favoring adoption of Octave in business
companies.



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Octave JIT consultancy

Richard Crozier

On 27/06/2019 08:16, roland65 wrote:
> Perhaps you should directly contact the guys at the llvm or libgccjit
> projects?
>

I was also going to suggest contacting the libgccjit dev who was already
working on integrating libgccjit with Octave. The only problem is that
libgccjit is alpha and like to have an unstable API for some time.

Richard
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Octave JIT consultancy

Daryl Maier
In reply to this post by Richard Crozier
Hello,

My name is Daryl Maier and I am one of the core contributors to the open
source Eclipse OMR compiler mentioned in this thread.  I was pleased to
discover your interest in possibly using the OMR compiler technology to
construct a JIT for Octave.  I encourage this effort, as the core of the
compiler was contributed into OMR precisely to make it easier to provide
JITs for language environments like this.

If you are not familiar with the OMR project please allow me to provide a
brief introduction.  OMR is a collection of open source, language agnostic
runtime components intended to be integrated into language runtimes where
that functionality is needed.  It provides core functionality for building
compilers (JITs) and garbage collectors, as well as infrastructural
components such as threading libraries and OS interfaces to make integration
and portability easier.  The runtime-neutral technology was distilled from
the J9 Java virtual machine, where it had over 15 years of development of
high-quality performance and memory optimizations and broad OS and processor
support.  The technology itself is very scalable, and has been used in small
embedded devices up through large mainframe environments.  It was open
sourced as an Eclipse Foundation project in 2016 under an Apache 2.0 and
Eclipse Public License v2 dual license.  Eclipse OMR lives in GitHub
(https://github.com/eclipse/omr) and is a very active project with a growing
community and near-daily contributions.

The compiler itself is written in C++ and was developed as a just-in-time
compiler from its inception.  Hence, its focus is on fast compile-time and
low memory footprint.  The compiler contains a rich suite of classical
compiler optimizations and the infrastructure to support speculative
optimizations and recompilation.  It can generate code for x86, Power, Z,
and AArch64 (in progress).  You can interface with the compiler component
either by writing a custom IL generator, or using a simplified programmatic
interface we've been developing called JitBuilder that abstracts away many
of the details of the compiler IL.  While the project's main consumer is
OpenJ9, we have had success producing proof-of-concept JITs for other
language environments such as Lua
(https://github.com/Leonardo2718/lua-vermelha) and WebAssembly
(https://github.com/wasmjit-omr/wasmjit-omr).

If anyone is interested in exploring the use of OMR for a JIT solution in
Octave feel free to reach out to me directly, or use one of the channels in
the Eclipse OMR community (either Slack via
https://join.slack.com/t/eclipse-omr/shared_invite/enQtMzg2ODIwODc4MTAyLTk4ZjJjNTZlZmMyMGRmYTczOTkzMGJiNTQ4NTA3YTA1NGU4MmJjNWI4NTBjOGNkNmNjMWQ3MmFmYjA4OGZjZjM
or GitHub).

I am perfectly happy to spend some time giving you a deeper dive on the
compiler technology, helping to understand the effort involved in creating a
JIT with it and learning more about Octave internals, guiding you to
determine the best way to integrate it into Octave, and answering any
questions you might have along the way.

Cheers,
..daryl




--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Octave JIT consultancy

GoSim
Hello,

I'm not an octave dev but interested in this topic. How would your program
deal with that octave variables can change type during their lifetimes?
Using your program, would it involve the current interpreter in any way? Or
would it be parallel to the interpreter? My guess is parallel which is why
no dev is answering you, maybe it is enough just to know the syntax?
How much work do you estimate it would be to create an m-file-compiler? In
lines of code. Can you describe the process?

ECU = electronic control unit, automotive industry
How do you assess the possibility to create a compiler from m-script to an
ECU with your program? There is a looot of money here....

kind regards




--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

RE: Fwd: Octave JIT consultancy

Daryl Maier

Hi,

I will try and answer some of your questions as best I can.  As I have limited knowledge of Octave internals (though I'm happy to be educated!) at this point some of my answers will be somewhat abstract.

>>> Using your program, would it involve the current interpreter in any way? Or would it be parallel to the interpreter?

It could work either way, but it depends on what the Octave interpreter is capable of doing.

Generally speaking, we have used the OMR compiler technology successfully both in environments where every method is compiled before execution and in mixed-mode environments that are a combination of interpreted and JITed methods.  Which technique a language environment chooses to use depends on the capabilities of the VM (for example, to support mixed-mode may require support for 4 different method transition types between methods: (I)nterp->Interp, I->(J)itted, J->I, and J->J, and on the performance objectives (for example, mixed mode is important if you care about the cost of compile-time and can't afford to compile every method).

The OMR compiler also supports multiple compilation threads which can run in parallel with your application and VM (including the interpreter).  This is how it is used in OpenJ9.  It also works just as well in a synchronous mode where methods are compiled on demand on the application thread before proceeding.  This is how the POC implementations in Lua and WebAssembly work, for instance.

>>> How would your program deal with that octave variables can change type during their lifetimes?

This would depend on the representation of the program as it is presented to the compiler.  Presumably, by the time it reaches the JIT, from the JIT's perspective it is dealing with different variables with different types.

>>> How much work do you estimate it would be to create an m-file-compiler? In lines of code. Can you describe the process?

It depends on what the starting point is.  However, for some perspective, the proof-of-concept Lua JIT I mentioned in my earlier email implements all but one of the PUC-Rio Lua VM internal opcodes in about 2500 LOC.  This includes using the JitBuilder interface to translate the internal Lua representation of a function into the IL expected by the OMR compiler, and glue logic in Lua to initiate JIT compilation and look up compiled targets.  Doing so yielded a number of noticeable performance improvements over the interpreter with a minimal amount of work.

The general process for any language environment looking to use OMR to compile is to first have some representation of the method that you wish to compile.  For instance, an AST (abstract syntax tree) with type information, or an intermediate representation of your method (for example, in bytecode form).  One could then use either JitBuilder or write your own custom "IL generator" to translate that representation into the internal tree-based, typed representation that OMR works with.

Deciding when to compile is another design decision, as you've alluded to in an earlier question.  Do you compile everything up front before the application executes (like a static compiler), or do you compile at runtime?  If you compile at runtime you will need to introduce some control logic to determine when and how a method is compiled as the program executes.  For example, in its simplest form, at the point where one method calls another you will need to check whether the target is compiled or not.  If it is compiled, then you dispatch to the compiled body.  If it is not compiled then you can either begin the process to initiate a compile, or do some other heuristic checks to defer that process (e.g., compile a method after 10 invocations).  This is an over-simplification of the process, but I think you get the idea.

Once you've decided when and what to compile, you then have to decide "how" to compile.  The OMR compiler allows you to define different "optimization strategies" that describe the optimizations you wish to apply to this compilation.  Some strategies may favour quick compiles while others may have more involved optimizations and consume more memory.  When the OMR compiler is used in OpenJ9, it uses a tiered-compilation approach and allows methods to be recompiled with increasing levels of optimization depending on the "hotness" of the method.

During the compilation process the JIT sometimes has questions that it needs answers to from the language "front end" in order to optimize or generate code (because language environments tend to be different, and each can have different rules or configurations).  For example, how do I fold a constant floating point expression at compile-time, what is the size of an address for the target machine, find me the address of a method given a method handle, etc.  Many of these queries have defaults, but a language environment would have to override them if necessary.  These are implemented in a "FrontEnd" interface and a compiler environment interface in OMR that you would have to provide.

The OMR compiler by default writes the generated code to an executable code buffer.  If the compilation was successful it simply returns the entry point address of the method just compiled.  At the moment, the language environment is responsible for managing these code addresses and providing a means to look up the compiled version of a method (though we may be introducing some infrastructure to generalize this in the near future).  There is also some less-tested (slightly above experimental) code that will permit the compiled functions to be persisted in an ELF representation for linking with other object files.

If you're interested in tinkering with this some more, last year at SPLASH'18 we ran a hands-on tutorial workshop that demonstrated how to integrate OMR into a WebAssembly environment.  You can find the tutorial here -> https://github.com/omr-turbo/wasmjit-omr/blob/turbo/turbo.md.  Although this is a simplified tutorial, it will give you a feel for what is involved in integrating your own compiler into an existing language environment.

>>> How do you assess the possibility to create a compiler from m-script to an ECU with your program? There is a looot of money here....

It might be possible to do what you're asking, but it's hard for me to say for sure without understanding more about both ends of the pipeline (i.e., the "m-script" end and the "ECU" end).  If the ECU end executes "instructions" like a CPU would then it should be possible to generate code for it.

Hope that helps.

Cheers,
..daryl


Inactive hide details for GoSim ---2019/09/16 01:06:26 PM---Hello,  I'm not an octave dev but interested in this topic. How wouGoSim ---2019/09/16 01:06:26 PM---Hello, I'm not an octave dev but interested in this topic. How would your program

From: GoSim <[hidden email]>
To: [hidden email]
Date: 2019/09/16 01:06 PM
Subject: [EXTERNAL] Re: Fwd: Octave JIT consultancy
Sent by: "Octave-maintainers" <octave-maintainers-bounces+maier=[hidden email]>





Hello,

I'm not an octave dev but interested in this topic. How would your program
deal with that octave variables can change type during their lifetimes?
Using your program, would it involve the current interpreter in any way? Or
would it be parallel to the interpreter? My guess is parallel which is why
no dev is answering you, maybe it is enough just to know the syntax?
How much work do you estimate it would be to create an m-file-compiler? In
lines of code. Can you describe the process?

ECU = electronic control unit, automotive industry
How do you assess the possibility to create a compiler from m-script to an
ECU with your program? There is a looot of money here....

kind regards




--
Sent from:
https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html 





Reply | Threaded
Open this post in threaded view
|

RE: Fwd: Octave JIT consultancy

GoSim
Daryl Maier wrote
>>>> How would your program deal with that octave variables can change type
> during their lifetimes?
>
> This would depend on the representation of the program as it is presented
> to the compiler.  Presumably, by the time it reaches the JIT, from the
> JIT's perspective it is dealing with different variables with different
> types.

Your program seems really nice. I would do a static compiler, i'm all about
keeping things simple.

Does this mean that octave has to present every variable as separate and
with different types to the compiler?

Octave uses gnu bison as parser: https://www.gnu.org/software/bison/
What do you think of making your software compatible with it?

kind regards




--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

RE: Fwd: Octave JIT consultancy

GoSim
In reply to this post by Daryl Maier
Your program seems really nice. I would do a static compiler, i'm all about
keeping things simple.

Does this mean that octave has to present every variable as separate and
with different types to the compiler?

Octave uses gnu bison as parser: https://www.gnu.org/software/bison/
What do you think of making your software compatible with it?

kind regards



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

RE: Fwd: Octave JIT consultancy

roland65
I found the following paper:
http://www.ee.columbia.edu/~marios/matlab/accel_matlab.pdf about the
Mathworks JIT compiler.

It gives some interesting (IMHO) information about the problem of dealing
with variable types.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

RE: Fwd: Octave JIT consultancy

GoSim
this is the interesting part:


> Run-time type analysis is based on the following prem-
> ise: If a line of M-code has been processed before, it is
> very likely that the variables have the same types and
> shapes that they had the last time the system saw this
> line. The first time that a line of code is executed, the
> system examines the variables and generates specific
> code for the data types and shapes that were found.
> Subsequent executions of the line can reuse this code
> as long as the system verifies that the variable types and
> sizes have not changed. Since the types rarely change,
> subsequent executions run as quickly as possible. If the
> types do change, the code is regenerated.

So it checks if the types changes. Easily done with a database of all
functions and all different input/output types. It seems to be clever enough
to not make this check on every line but just on lines where type changes is
possible. I prefer not allowing the user to change variable types and give
an error, but mathworks' solution is also nice.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html