Octave and Intel MKL

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Octave and Intel MKL

Felix Willenborg
Dear all,

I'm trying to compile octave (additionally to ATLAS) with Intel MKL and 64-bit indexing. Intel offers a guide where they try to give instructions on how to compile [1] (please notice Remi Payans comment in the comment section). Unfortunately they're not working (for me).

When trying to build with the Intel Compiler, the following happens:
./configure --disable-readline --prefix=/cluster/programs/octave/4.2.1-mkl --enable-64 --with-blas="-Wl,--start-group -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -liomp5 -lpthread -lm" --with-lapack="-Wl,--start-group -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -liomp5 -lpthread -lm" --with-fftw3="-Wl,--start-group -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -liomp5 -lpthread -lm" --with-fftw3f="-Wl,--start-group -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -liomp5 -lpthread -lm" LD_LIBRARY_PATH='/cluster/programs/octave/4.2.1-mkl/lib:$MKLROOT/lib/intel64:$MKLROOT/../compiler/lib/intel64:$LD_LIBRARY_PATH' LIBRARY_PATH='/cluster/programs/octave/4.2.1-mkl/lib:$LIBRARY_PATH' CPPFLAGS='-I/cluster/programs/octave/4.2.1-mkl/include -I$MKLROOT/include -I$MKLROOT/include/fftw' LDFLAGS='-L/cluster/programs/octave/4.2.1-mkl/lib -L$MKLROOT/lib/intel64 -L$MKLROOT/../compiler/lib/intel64' CC=icc CXX=icc F77=ifort CFLAGS="-O3 -ipo- -fPIC -std=c99 -DMKL_ILP64 -DM_PI=3.1415926535897932384" F77_INTEGER_8_FLAG='-i8'
[...]
checking for dummy main to link with Fortran 77 libraries... none
checking for Fortran 77 name-mangling scheme... lower case, underscore, no extra underscore
configure: error: to build Octave, the C++ to Fortran calling convention must be known.

With --enable-fortran-calling-convention I can specify the calling conventions f2c, visual-fortran, cray and gfortran. All options fail with
[...]
configure: error: to build Octave with 64-bit indexing support your Fortran compiler must have an option for setting the default integer size to 8 bytes.  See the file INSTALL for more information.

which is very likely attributable to the fact of wrong calling conventions (maybe somebody has here an idea). When trying Remi Payans suggestion from the comment section using the gnu compiler with mkl libraries for gnu threading
./configure --disable-readline --enable-64 --prefix=/cluster/programs/octave/4.2.1-mkl --with-blas="-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread" --with-lapack="-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread" --with-fftw3="-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread -lm" --with-fftw3f="-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread -lm" LD_LIBRARY_PATH='/cluster/programs/octave/4.2.1-mkl/lib:$MKLROOT/lib/intel64:$MKLROOT/../compiler/lib/intel64:$LD_LIBRARY_PATH' LIBRARY_PATH='/cluster/programs/octave/4.2.1-mkl/lib:$LIBRARY_PATH' CPPFLAGS='-I/cluster/programs/octave/4.2.1-mkl/include -I$MKLROOT/include -I$MKLROOT/include/fftw' LDFLAGS='-L/cluster/programs/octave/4.2.1-mkl/lib -L$MKLROOT/lib/intel64 -L$MKLROOT/../compiler/lib/intel64' CFLAGS="-O2 -fPIC -DMKL_ILP64 -DM_PI=3.1415926535897932384" FFLAGS="-O2 -fPIC" F77=gfortran
[...]
checking fftw3.h usability... yes
checking fftw3.h presence... yes
checking for fftw3.h... yes
checking for fftw_plan_dft_1d in -lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread -lm... no
configure: WARNING: FFTW3 library not found.  The slower FFTPACK library will be used instead.
checking for fftw3.h... (cached) yes
checking for fftwf_plan_dft_1d in -lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread -lm... no
configure: WARNING: FFTW3F library not found.  The slower FFTPACK library will be used instead.
[...]
configure: error: BLAS doesn't seem to support 64-bit integers.  This is incompatible with --enable-64.

This is also not working when taking MKL_LP64 libraries and removing --enable64. Somehow he can not find the needed numerical functions. There I get the same messages with the difference that -lblas and -llapack are working (which was expectable).

Has anybody gathered experience in that topic and can give me the desired clue in order to compile it with mkl? Looking forward to your answers.

Best wishes,
Felix Willenborg

[1]: https://software.intel.com/en-us/articles/using-intel-mkl-in-gnu-octave
-- 
Felix Willenborg

Arbeitsgruppe Machine Learning und Exzellenzcluster Hearing4all
Department für Medizinische Physik und Akustik
Fakultät für Medizin und Gesundheitswissenschaften 
Carl von Ossietzky Universität Oldenburg

Küpkersweg 74, 26129 Oldenburg
Tel: +49 441 798 3945

https://www.uni-oldenburg.de/machine-learning/

_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: Octave and Intel MKL

Mike Miller-4
On Tue, Sep 12, 2017 at 23:50:16 +0200, Felix Willenborg wrote:
> I'm trying to compile octave (additionally to ATLAS) with Intel MKL and
> 64-bit indexing.
[…]
> This is also not working when taking MKL_LP64 libraries and removing
> --enable64. Somehow he can not find the needed numerical functions.
> There I get the same messages with the difference that -lblas and
> -llapack are working (which was expectable).

I don't have any particular experience, but I would recommend you start
with the simplest configuration, using GCC with 32-bit Fortran integers
and the MKL LP64 library. You might also want to test with the Octave
development branch, which has reworked a lot of configure and code
concerning the C/Fortran interface.

I think the following should work

    ./configure \
      --with-blas='-lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -liomp5' \
      LDFLAGS='-L/$intel/mkl/lib/intel64 -L/$intel/lib/intel64' \
      LD_LIBRARY_PATH='/$intel/mkl/lib/intel64:/$intel/lib/intel64'

If that doesn't work, then config.log should help you figure out why.

If you do get that working, then try moving on to ILP64.

--
mike

_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Octave and Intel MKL

Felix Willenborg
Hey Mike,

thanks for your advice. I could compile octave 4.2.1 from source with Intel MKL (Intel Parallel Studio XE 2016 (update3.120)). With this (big) configuration call it was possible:
./configure --disable-readline --without-curl --enable-docs=no --prefix=/cluster/programs/octave/4.2.1-mkl --with-blas='-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5' --with-lapack='-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5' --with-fftw3='-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lm' --with-fftw3f='-lmkl_gf_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lm' LD_LIBRARY_PATH='/cluster/compilers/intel/parallel_studio_xe_2016_update3/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/:/cluster/compilers/intel/parallel_studio_xe_2016_update3/compilers_and_libraries_2016.3.210/linux/mkl/../compiler/lib/intel64/:$LD_LIBRARY_PATH' CPPFLAGS='-I/cluster/compilers/intel/parallel_studio_xe_2016_update3/compilers_and_libraries_2016.3.210/linux/mkl/include/ -I/cluster/compilers/intel/parallel_studio_xe_2016_update3/compilers_and_libraries_2016.3.210/linux/mkl/include/fftw/' LDFLAGS='-L/cluster/compilers/intel/parallel_studio_xe_2016_update3/compilers_and_libraries_2016.3.210/linux/mkl/lib/intel64/ -L/cluster/compilers/intel/parallel_studio_xe_2016_update3/compilers_and_libraries_2016.3.210/linux/mkl/../compiler/lib/intel64/' CC=gcc CXX=gcc F77=gfortran CFLAGS='-O3 -fPIC -DMKL_ILP64 -DM_PI=3.1415926535897932384 -lstdc++' FFLAGS='-O3 -fPIC' F77_INTEGER_8_FLAG='-finteger-4-integer-8'
My initial problem was that I had a terminal variable in the LDFLAGS (like -L$MKLROOT/lib/intel64). In Intels manual for compiling octave with Intel MKL [2] they exported the variable for LDFLAGS whereas I locally defined it which was my mistake (dummy). Therefore I defined the whole path in my local definition. After that, I got many errors at the build step
CXXLD    src/mkoctfile
containing many many missing definitions of std::string. After doing some research I found out that ann additional -lstdc++ solved that problem but I don't get why.

In one of my earlier e-mails here I introduced a pseudo 'MFLOP/S' script [1] in order to roughly measure the efficiency. Compared to another build of mine with OpenBLAS and 64-bit indexing I got
MFLOPS: (23.733844 +- 8.549004) (MKL ILP64)
MFLOPS: (24.997268 +- 5.954096) (MKL LP64)
MFLOPS: (29.056399 +- 0.131886) (OpenBLAS)

I was wondering if that could be or if I build it wrong? I expected MKL to be the fastest of all numerical libraries out there and am surprised that OpenBLAS is much better than all others. You guys got any thought on that?

Best wishes,
Felix Willenborg

[1]: http://lists.gnu.org/archive/html/help-octave/2017-09/msg00084.html
[2]: https://software.intel.com/en-us/articles/using-intel-mkl-in-gnu-octave

Am 13.09.2017 um 18:55 schrieb Mike Miller:
On Tue, Sep 12, 2017 at 23:50:16 +0200, Felix Willenborg wrote:
I'm trying to compile octave (additionally to ATLAS) with Intel MKL and
64-bit indexing.
[…]
This is also not working when taking MKL_LP64 libraries and removing
--enable64. Somehow he can not find the needed numerical functions.
There I get the same messages with the difference that -lblas and
-llapack are working (which was expectable).
I don't have any particular experience, but I would recommend you start
with the simplest configuration, using GCC with 32-bit Fortran integers
and the MKL LP64 library. You might also want to test with the Octave
development branch, which has reworked a lot of configure and code
concerning the C/Fortran interface.

I think the following should work

    ./configure \
      --with-blas='-lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -liomp5' \
      LDFLAGS='-L/$intel/mkl/lib/intel64 -L/$intel/lib/intel64' \
      LD_LIBRARY_PATH='/$intel/mkl/lib/intel64:/$intel/lib/intel64'

If that doesn't work, then config.log should help you figure out why.

If you do get that working, then try moving on to ILP64.



_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave

-- 
Felix Willenborg

Arbeitsgruppe Machine Learning
Department für Medizinische Physik und Akustik
Fakultät für Medizin und Gesundheitswissenschaften 
Carl von Ossietzky Universität Oldenburg

Küpkersweg 74, 26129 Oldenburg
Tel: +49 441 798 3945

https://www.uni-oldenburg.de/machine-learning/

_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Octave and Intel MKL

Mike Miller-4
On Sun, Sep 17, 2017 at 20:33:35 +0200, Felix Willenborg wrote:
> whole path in my local definition. After that, I got many errors at the
> build step
>
>     CXXLD    src/mkoctfile
>
> containing many many missing definitions of std::string. After doing
> some research I found out that ann additional -lstdc++ solved that
> problem but I don't get why.

You should use CXX=g++ instead of CXX=gcc to fix this.

--
mike

_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Octave and Intel MKL

Felix Willenborg
Uff oh wow, thanks for the hint. How embarassing.

Best wishes,
Felix

Am 17.09.2017 um 21:07 schrieb Mike Miller:
On Sun, Sep 17, 2017 at 20:33:35 +0200, Felix Willenborg wrote:
whole path in my local definition. After that, I got many errors at the
build step

    CXXLD    src/mkoctfile

containing many many missing definitions of std::string. After doing
some research I found out that ann additional -lstdc++ solved that
problem but I don't get why.
You should use CXX=g++ instead of CXX=gcc to fix this.



_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave

_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave

signature.asc (836 bytes) Download Attachment