Can Otave (Matlab code) be added to C++ as a library?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Can Otave (Matlab code) be added to C++ as a library?

Pablo Fernández
Can Otave (Matlab code) be added to C++ as a library?

Best regards



Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

siko1056
On 3/30/20 10:49 PM, Pablo Fernández wrote:
> Can Otave (Matlab code) be added to C++ as a library?
>
> Best regards
>

In general not, Octave is an interpreted language.  Depending on your
purpose, you can call Octave code from your C++ program as described in [1].

HTH,
Kai


[1] https://octave.org/doc/v5.2.0/Standalone-Programs.html


Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

evansste
The link, about standalone programs, mentions that Octave has an archived
library file called "liboctave.a".  Where would I find this file?  Here's
that link, which was posted earlier by siko1056:

https://octave.org/doc/v5.2.0/Standalone-Programs.html 

I've written programs in Fortran, and would like to use "ismember" and
"unique", which are native Octave functions, in my Fortran programs.  I
already have many of my own Fortran programs stored in an archive file.
Could I just include the "liboctave.a" file, along with my own archive file,
and be able to use Octave's native functions?

For example, this is how I currently compile a Fortran program so that I may
use my own Fortran functions in the program:

caf <fortran_source_file.f08> -o <executable_name_of_source_file>
fortranlib.a

If I want to also include Octave's functions would I simply do something
like:

caf <fortran_source_file.f08> -o <executable_name_of_source_file>
fortranlib.a liboctave.a

I'd love to use native Octave functions in my Fortran programs since doing
so would prevent me from having to re-write those functions in Fortran.
Would this work, and where might I find the "liboctave.a" file?

Thanks so much for your time.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

siko1056
On 4/8/20 6:40 AM, evansste wrote:

> The link, about standalone programs, mentions that Octave has an archived
> library file called "liboctave.a".  Where would I find this file?  Here's
> that link, which was posted earlier by siko1056:
>
> https://octave.org/doc/v5.2.0/Standalone-Programs.html 
>
> I've written programs in Fortran, and would like to use "ismember" and
> "unique", which are native Octave functions, in my Fortran programs.  I
> already have many of my own Fortran programs stored in an archive file.
> Could I just include the "liboctave.a" file, along with my own archive file,
> and be able to use Octave's native functions?
>
> For example, this is how I currently compile a Fortran program so that I may
> use my own Fortran functions in the program:
>
> caf <fortran_source_file.f08> -o <executable_name_of_source_file>
> fortranlib.a
>
> If I want to also include Octave's functions would I simply do something
> like:
>
> caf <fortran_source_file.f08> -o <executable_name_of_source_file>
> fortranlib.a liboctave.a
>
> I'd love to use native Octave functions in my Fortran programs since doing
> so would prevent me from having to re-write those functions in Fortran.
> Would this work, and where might I find the "liboctave.a" file?
>
> Thanks so much for your time.
>


By default Octave is build without static libraries, thus all major
Linux distributions do not provide these files by default either.  One
possibility is to compile Octave yourself using

  ./configure --enable-static

Another way is to link the dynamic Octave libraries.  I don't have
OpenCoarrays to test, but maybe this one might work:

  caf <fortran_source_file.f08> -o <executable_name_of_source_file>
  fortranlib.a -L/path/to/liboctave -loctave

HTH,
Kai


Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

Carlo de Falco-2
In reply to this post by evansste


> Il giorno 7 apr 2020, alle ore 23:40, evansste <[hidden email]> ha scritto:
>
> The link, about standalone programs, mentions that Octave has an archived
> library file called "liboctave.a".  Where would I find this file?  Here's
> that link, which was posted earlier by siko1056:
>
> https://octave.org/doc/v5.2.0/Standalone-Programs.html 
>
> I've written programs in Fortran, and would like to use "ismember" and
> "unique", which are native Octave functions, in my Fortran programs.  I
> already have many of my own Fortran programs stored in an archive file.
> Could I just include the "liboctave.a" file, along with my own archive file,
> and be able to use Octave's native functions?
>
> For example, this is how I currently compile a Fortran program so that I may
> use my own Fortran functions in the program:
>
> caf <fortran_source_file.f08> -o <executable_name_of_source_file>
> fortranlib.a
>
> If I want to also include Octave's functions would I simply do something
> like:
>
> caf <fortran_source_file.f08> -o <executable_name_of_source_file>
> fortranlib.a liboctave.a
>
> I'd love to use native Octave functions in my Fortran programs since doing
> so would prevent me from having to re-write those functions in Fortran.
> Would this work, and where might I find the "liboctave.a" file?
>
> Thanks so much for your time.
>


Some years ago I wrote a short example on the Octave wiki about how to invoke Octave liboctave
methods from a fortran program :

https://wiki.octave.org/Fortran

This was based on an old Octave release so a few adjustments would be needed for it to work with
Octave 5.x or 6.x ...

Is this similar to what you had in mind?

c.





Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

evansste
Thanks to both of you.

To answer the question that "Carlo de Falco-2" asked -- yes, your link is
similar to what I'm trying to do.  The following link is also similar, since
it shows a compiled, yet C++, program running a native Octave function.

https://octave.org/doc/v5.2.0/Standalone-Programs.html

I've been trying to find a way to do this without having to use C++.  I
recognize that Octave uses LAPACK and BLAS, and that those subroutines are
written in Fortran.  I also ran across the following link:

https://www.math.utah.edu/software/c-with-fortran.html

It's very informative, and gives a good explanation as to why working with
mixed programming languages can be problematic, and why programmers are
still stuck with the reality of often resorting to it.  If I could use
native Octave functions without having to also include C++, that would be a
huge bonus.

It seems as if including C++, in my Fortran programs, would force me to pass
information using files.  I've been implementing an extremely clunky version
of that, up til now.  Portions of the programs are written in Octave, but
using files to implement Fortran.  It's been a migration process, since the
goal is to completely convert the program to Fortran, for the sake of speed.
However, all speed is lost, and the program ends up being even slower, when
I use my Fortran versions of "ismember" and "unique", without using LAPACK
and BLAS.

As "siko1056" pointed out, it may be possible to get a static library by
compiling Octave with the appropriate static option.  I've recently learned
that the Octave library would then be pre-compiled, and in machine language.
Because of this, shouldn't it be theoretically possible to use the functions
directly with Fortran, or any other language, or am I wrong about that?

I don't know how Octave is using LAPACK and BLAS to make its native
functions so fast, but that's clearly what I also need to do.  However, I'm
not quite sure where I'd begin with that option.  Linear Algebra techniques
appear to have nothing to do with matrix manipulation, which is what
"ismember" does.  So I can't begin to think of what the algorithm would be.

Because I'm not a seasoned programmer, I've been considering the idea of
hiring a programmer to write a Fortran version of "ismember", using LAPACK
and BLAS so that it can remain fast.  Is that feasible, or is the program so
complicated that I'd probably go broke going down that road?

Thanks again, for your input.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

michaell
Two answers: first, both "ismember" and "unique" are m-code functions. Thus,
in order to use them you need the whole interpreter. This is also possible,
but calling it from fortran would be even more effort than from C++. Of
course, under the hood it is "sort" that does the work, and that is a
compiled function. So if you could transform your code to use "sort", you
would not need the interpreter.

And that is then the second answer: no, of course it is not matrix
manipulation as implemented by LAPACK and BLAS that you want here, but it is
just sorting. Octave's sort is taken straight from some python
implementation, by the way. But even plain C has a sorting function (qsort)
in its standard library, C++ will have lots of them, and probably also
Fortran has some sorting function in its standard library (and if not, you
could definitely find one in a permissive licence on the web).

As to how this is used for "ismember" and "unique": unique is just that you
sort your input array (takes O(N*log(N)) time and O(N) space) and step
through it and take only the elements that are different from the element
before (takes O(N) time and additional O(1) space). As regards ismember: if
it is just to test once whether a single element x is a member of a large
vector S, nothing will beat the trivial solution of stepping through S and
testing whether x is equal to S(i). This would be even much faster than
Octave does it. Also if you have a small vector A of, say, N_A=10 elements,
and you want to test which of these are in S, where S is large, say N_S=one
million elements, the trivial solution will be the best. However, this
scales with O(N_S*N_A). If A is larger, at some point it will become
beneficial to first sort S (takes O(N_S*log(N_S))), and then test the
elements by bisection (takes O(N_A*log(N_S))). However, if N_A is much
larger than 1 but still much smaller than N_S, it would be even better to
sort A and step through the elements of S. Another possibility that could be
competitive if N_A and N_S are the same order of magnitude is to sort both A
and S, and then step through A and S simultaneously: if A(i)<S(j) increment
i, if A(i)>S(j) increment j, and if they are equal, you know that A(i) is in
S, and you increment both.

So you see, you can definitely be much faster than Octave can ever be,
because you know about your specific situation, while Octave's algorithms
have to have acceptable behaviour in all situations. For instance, it could
be that you can generate A or S so that they are sorted all the time. Or
perhaps you test not just once, but many times, in which case you can keep
the ordering.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

evansste
Thanks so much, Michaell, for all of that information.

The matricies, which are inputs for "ismember", each contain over a million
rows, and the number of rows is identical for each input.  So I wrote a
program which employs the last idea that you mentioned.

I'm happy to say that it was way faster than any of the other algorithms
I've tried.  In order for it to work, I had to find a sortrows function.  I
was able to find one, but it takes a lot longer than the actual comparison
part of the algorithm.  The program takes about twenty minutes to sort each
of the inputs.  However, when passing pre-sorted inputs, I've found that the
actual comparison portion only takes 0.3 seconds.  This is the same amount
of time that it takes to run Octave's "ismember" function on the
non-presorted set.  So unless I find a sortrows algorithm which literally
takes no time, it will still be slower than Octave's implementation.
However, this is certainly the closest I've gotten to actually solving this
problem and implementing a solution that doesn't take nearly an hour to run,
or longer.

I want to thank you for sharing these algorithms with me.  The last one is
actually fairly simple, but it never would have occurred to me -- not for a
moment.

Even though the sortrows algorithm, which I found online, takes about twenty
minutes to sort each of the inputs, it's still a lot faster than a previous
version, which I wrote.  My version uses a method of swapping neighboring
entries after comparing them to see if a swap is necessary.  I'm sure you're
familiar with the procedure.

Here's the much faster version, which I was able to find online.

https://stackoverflow.com/questions/35324153/sorting-arrays-by-rows-in-fortran

It, too, uses an algorithm that I never would have thought of.

I still don't know exactly what Octave is doing in order to perform the
entire "ismember" operation so quickly.  They may be using a similar
approach, with an infinitely fast sort method.  However that method would
have to be instantaneous in order to offer the performance that it does.

In any case, I feel a whole lot closer to getting this to work.  So, I
greatly appreciate the valuable input you've given.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

evansste
In reply to this post by michaell
I was able to find a sorting algorithm which completes in a little over a
second, rather than twenty minutes.  So now, I have a Fortran "ismember"
function which completes in 2.5 seconds.  This time isn't quite as fast as
Octave's, but it may be acceptable for my needs.

Thanks to everyone who contributed on this.  I learned a lot, and you
certainly contributed to my success.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Can Octave (Matlab code) be added to C++ as a library?

michaell
As to how Octave does it: for m-code functions, it is easily found out: you
say "help ismember", and in the top line you see the file that implements
the function. And then, if you can read m-code, you just read that function.
Beware: Octave is published under the GPL, so if you ever would want to (i)
publish your code and (ii) under a different licence than the GPL, you
should rather not do that. But I can definitely tell you the idea: It is
just my suggestion where you sort one of the arrays (always the second one)
and do a lookup by bisection.

It should not take 20 minutes to sort a million entries (and a sortrows
should not be much more effort than just a sort). This sounds like a
O(N^2)-algorithm, and what you linked to indeed is. The best general purpose
sorting algorithms are O(N*log(N)), and they do not need to be very
complicated. The algorithm Octave uses is quite sophisticated, but this
sophistication is to a large part just to run very good in a number of
scenarios (presorted data, unsorted data, reversely sorted data, partially
sorted data and so on). If you know for example that your values are
uniformly distributed over an interval and practically perfectly random, you
could use a very simple quicksort where you use bisection of the interval in
all but the last stages instead of the typical choice of the first entry as
pivot (this would practically be the optimum pivot, thus giving you less
stages). And you get a sortrows out of that by comparing the rows in the
lexicographic sense. I am quite certain that you should be able to write
something like that in twenty lines of code, that will be faster than Octave
can ever be (again, because you know your situation).

And yes, generally it can be stated that if you use an O(N*log(N)) algorithm
for sorting, it will be "infinitely fast" in the sense that it will always
take much more time to get the entries into RAM than to sort them.





--
Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html