Unofficial octave 4.4.0 Windows binary with large arrays

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Unofficial octave 4.4.0 Windows binary with large arrays

tmacchant
I have uploaded Unofficial octave 4.4.0 Windows binary with large arrays ().

http://www.tatsuromatsuoka.com/octave/Eng/Win/


octave-4.4.0-w64-LA.7z


Tatsuro


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

mmuetzel
I thought that the 64-bit version of Octave now supported large arrays by
default. At least that is how I understood the NEWS file:
"** On systems with 64-bit pointers, --enable-64 is now the default and
    Octave always uses 64-bit indexing.  However, if the configure
    script determines that the BLAS library uses 32-bit integers, then
    operations using the following libraries are limited to arrays with
    dimensions that are smaller than 2^31 elements:

      BLAS  LAPACK  QRUPDATE  SuiteSparse  ARPACK

    Additionally, the following libraries use "int" internally, so
    maximum problem sizes are always limited:

      glpk  Qhull"

But I now see that the official Windows build was configured with
"--disable-64". Was that intended?



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

tmacchant
----- Original Message -----

> From: mmuetzel
> To: help-octave
> Cc:
> Date: 2018/5/6, Sun 03:41
> Subject: Re: Unofficial octave 4.4.0 Windows binary with large arrays
>
> I thought that the 64-bit version of Octave now supported large arrays by
> default. At least that is how I understood the NEWS file:
> "** On systems with 64-bit pointers, --enable-64 is now the default and
>     Octave always uses 64-bit indexing.  However, if the configure
>     script determines that the BLAS library uses 32-bit integers, then
>     operations using the following libraries are limited to arrays with
>     dimensions that are smaller than 2^31 elements:
>
>       BLAS  LAPACK  QRUPDATE  SuiteSparse  ARPACK
>
>     Additionally, the following libraries use "int" internally, so
>     maximum problem sizes are always limited:
>
>       glpk  Qhull"

What you wrote is true.

> But I now see that the official Windows build was configured with
> "--disable-64". Was that intended?

How did you see "But I now see that the official Windows build was configured with
 "--disable-64"." ?

I noticed the fact that official windows 64bit binary does not support large arrays
by the test  shown in the below
>> a = zeros (1024*1024*1024*3, 1, 'int8');

https://wiki.octave.org/Enable_large_arrays:_Build_octave_such_that_it_can_use_arrays_larger_than_2Gb.

Anyway, as you know, windows binary is build using mxe-octave on linux.
For stable source (now it is 4.4.0 source), mxe-octave should be configured if one want to build octave with
large arrays.
--enable-64 --enable-fortran-int64.

I suspect that official 64 bit binary is built using mxe-octave without the above options.

Tatsuro


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

mmuetzel
One of the flags in __octave_config_info__ ("config_opts") when called in the
official Octave 4.4.0 (Windows installer 64-bit) is '--disable-64'.
For my own cross-compiled versions there is '--enable-64' even if I don't
specifically configure mxe-octave with --enable-64. The configure flags I
use for mxe-octave are (default Octave that is):
./configure --enable-devel-tools --enable-windows-64 --enable-octave=default
--enable-binary-packages
--with-pkg-dir=/home/markus/Repositories/Octave/mxe-octave-pkg --with-ccache
--disable-system-opengl

Is it possible to somehow determine which flags where used to configure
mxe-octave for the official installer?

And still: Is it intentional that the official Windows 64-bit versions use
32-bit indexing?

Markus



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

mmuetzel
Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

tmacchant
In reply to this post by mmuetzel


> From: mmuetzel > To: [hidden email]
> Cc:
> Date: 2018/5/6, Sun 18:28
> Subject: Re: Unofficial octave 4.4.0 Windows binary with large arrays
>
> One of the flags in __octave_config_info__ ("config_opts") when called
> in the
> official Octave 4.4.0 (Windows installer 64-bit) is '--disable-64'

Thanks for information.

.
> For my own cross-compiled versions there is '--enable-64' even if I
> don't
> specifically configure mxe-octave with --enable-64. The configure flags I
> use for mxe-octave are (default Octave that is):
> ./configure --enable-devel-tools --enable-windows-64 --enable-octave=default
> --enable-binary-packages
> --with-pkg-dir=/home/markus/Repositories/Octave/mxe-octave-pkg --with-ccache
> --disable-system-opengl

There are differences between stable (4.4.0) and defalut (4.5.0).

For 4.4 as you wrote. BLAS etc libraries. should be prepared with 64 bit index.
For mxe-octave for stable (4.4)  --enable-fortran-int64 ineviably required 
for 64 bit index build for stable

For 4,5, one can use 64 bit index evenif BLAS etc. are built with 32bit index.
Thus  --enable-fortran-int64 is not required for default (4.5) mxe build.


> Is it possible to somehow determine which flags where used to configure
> mxe-octave for the official installer?


--enable64 --enable-windows64 and --enable-fortran-int64 for stable branch.
 


> And still: Is it intentional that the official Windows 64-bit versions use
> 32-bit indexing?
>


Perhaps 64 index for 64 bit binary will be default in the next major release.
I am not a developer but a one of heavy users.  
I suspect that windows binary this release is built with 32 bit index is intentional.

Developer will make a definite rely.

Tatsuro



Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

mmuetzel
> > Is it possible to somehow determine which flags where used to configure
> > mxe-octave for the official installer?
>
>
> --enable64 --enable-windows64 and --enable-fortran-int64 for stable
> branch.

If those are the configure flags used for the official installer, it is even
stranger how it ended up with "--disable-64" in its configure flags...

I think I never cross-compiled stable Octave. But it was definitely
different for 4.3.0+ and still is for 5.0.0.



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

tmacchant
----- Original Message -----

> From: mmuetzel 
> To: help-octave> Cc:
> Date: 2018/5/6, Sun 19:17
> Subject: Re: Unofficial octave 4.4.0 Windows binary with large arrays
>
>>  > Is it possible to somehow determine which flags where used to
> configure
>>  > mxe-octave for the official installer?
>>
>>
>>  --enable64 --enable-windows64 and --enable-fortran-int64 for stable
>>  branch.
>
> If those are the configure flags used for the official installer, it is even
> stranger how it ended up with "--disable-64" in its configure flags...
>
> I think I never cross-compiled stable Octave. But it was definitely
> different for 4.3.0+ and still is for 5.0.0.
>
>
>


Sorry, I have partly misled what you say.

For 4.4, to build flawless octave windows binary with 64bit index, we need
--enable-fortran-int64 for mxe.

Without  --enable-fortran-int64, build with 64 bit index gives broken binary.

Two options exist for 64 bit binary for  mxe cross build for windows of octave 4,4.
1. disable 64 bit index
2. enable 64 bit index and other libraires are also buit 64 bit index

For case 2, other libraires are also buit 64 bit index, but it has not well tesed at the moment,


Perhaps thus the developer selected the first.

Again this is mere my gnuess.

Tatsuro


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

mmuetzel
> Without  --enable-fortran-int64, build with 64 bit index gives broken
binary.

What do you mean by "broken binary"?

This is an excerpt of the config.log of my mxe-octave:
ENABLE_64='yes'
ENABLE_BINARY_PACKAGES='yes'
ENABLE_DEP_DOCS='no'
ENABLE_DEVEL_TOOLS='yes'
ENABLE_DOCS='yes'
ENABLE_FORTRAN_INT64='no'
ENABLE_JAVA='yes'
ENABLE_JIT='no'
ENABLE_LIB64_DIRECTORY='no'
ENABLE_OCTAVE='default-'
ENABLE_OPENBLAS='yes'
ENABLE_QHULL_NO_STRICT_ALIASING_FLAG='no'
ENABLE_QT5='yes'
ENABLE_WINDOWS_64='yes'

If I am reading this correctly, I don't build 64-bit Fortran libraries but
do have 64-bit indexing. A cross-build with these settings doesn't fail any
of the tests in the test suite (except some xfails).

I am not aware of any changes in the default branch since it was merged to
stable for 4.4 that should have made a difference.

btw: I am very happy that you provide a modified version of Octave for
Windows that allows using large arrays. I am just wondering why the
"official" installer doesn't support them.




--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

tmacchant
----- Original Message -----

> From: mmuetzel 
> To: help-octave> Cc:
> Date: 2018/5/6, Sun 20:09
> Subject: Re: Unofficial octave 4.4.0 Windows binary with large arrays
>
>>  Without  --enable-fortran-int64, build with 64 bit index gives broken
> binary.
>
> What do you mean by "broken binary"?

http://octave.1599824.n4.nabble.com/Octave-on-Windows-with-large-arrays-td4682366.html

I have met the phenomena when tried octave 4.2.1 for windows with 64 index.
At first I built without --enable-fortran-int64.
That gave many errors for __run_test_suite__ and could not be used practically.
 


> This is an excerpt of the config.log of my mxe-octave:
> ENABLE_64='yes'
> ENABLE_BINARY_PACKAGES='yes'
> ENABLE_DEP_DOCS='no'
> ENABLE_DEVEL_TOOLS='yes'
> ENABLE_DOCS='yes'
> ENABLE_FORTRAN_INT64='no'
> ENABLE_JAVA='yes'
> ENABLE_JIT='no'
> ENABLE_LIB64_DIRECTORY='no'
> ENABLE_OCTAVE='default-'
> ENABLE_OPENBLAS='yes'
> ENABLE_QHULL_NO_STRICT_ALIASING_FLAG='no'
> ENABLE_QT5='yes'
> ENABLE_WINDOWS_64='yes'
>
> If I am reading this correctly, I don't build 64-bit Fortran libraries but
> do have 64-bit indexing. A cross-build with these settings doesn't fail any
> of the tests in the test suite (except some xfails).
>
> I am not aware of any changes in the default branch since it was merged to
> stable for 4.4 that should have made a difference.

See jwe's statements in the thread. 
http://octave.1599824.n4.nabble.com/Octave-on-Windows-with-large-arrays-tp4682366p4682460.html

The difference between stable and default in the mxe octave is in the internal code of octave.
we could not find any differences in configure checks. 

> btw: I am very happy that you provide a modified version of Octave for
> Windows that allows using large arrays. I am just wondering why the
> "official" installer doesn't support them.

I perhaps forgot to add --enable-devel-tools for mxe-octave.
I will check that and if I had make mistake, i rebuild and re-uploade 
corrected binary. 

> I am just wondering why the
> "official" installer doesn't support them.

I do not know.
I added address jwe (chief developper) and John D (the most working person on mxe octave)
 to the send list of the mail.

I hope that they will answer to your quesstion.

Tatsuro


Reply | Threaded
Open this post in threaded view
|

Re: Unofficial octave 4.4.0 Windows binary with large arrays

mmuetzel
> See jwe's statements in the thread.
> http://octave.1599824.n4.nabble.com/Octave-on-Windows-with-large-arrays-tp4682366p4682460.html

> The difference between stable and default in the mxe octave is in the
> internal code of octave.

The default branch from back then (4.3.0+) is the stable branch now (4.4.0).
So these options (--enable-64 and --enable-fortran-int64) now apply both to
default and to stable Octave.
It should now be possible to combine --enable-64 with
--disable-fortran-int64. At least this combination works for me with the
default branch since quite a few months. And it should also work for the
current stable branch (4.4.0).

> > I am just wondering why the
> > "official" installer doesn't support them.
>
> I do not know.
> I added address jwe (chief developper) and John D (the most working person
> on mxe octave)
>  to the send list of the mail.
>
> I hope that they will answer to your quesstion.

Right. They probably know best why the official installer for Windows ships
an Octave configured with --disable-64.



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Re: Unofficial octave 4.4.0 Windows binary with large arrays

John W. Eaton
Administrator
In reply to this post by mmuetzel
On 05/05/2018 02:41 PM, mmuetzel wrote:
> I thought that the 64-bit version of Octave now supported large arrays by
> default.

It does.

> But I now see that the official Windows build was configured with
> "--disable-64". Was that intended?

No, that was a mistake.

I'll upload a build with the correct 64-bit indexing and 32-bit BLAS and
other Fortran libraries soon.  This configuration is the same as is
typical for all other systems where the system BLAS, LAPACK, and other
Fortran libraries are built with the default 32-bit INTEGER type.

jwe



Reply | Threaded
Open this post in threaded view
|

Re: Re: Unofficial octave 4.4.0 Windows binary with large arrays

tmacchant
----- Original Message -----

> From: John W. Eaton
> To: mmuetzel ; help-octave
> Cc:
> Date: 2018/5/7, Mon 02:47
> Subject: Re: Re: Unofficial octave 4.4.0 Windows binary with large arrays
>
> On 05/05/2018 02:41 PM, mmuetzel wrote:
>>  I thought that the 64-bit version of Octave now supported large arrays by
>>  default.
>
> It does.
>
>>  But I now see that the official Windows build was configured with
>>  "--disable-64". Was that intended?
>
> No, that was a mistake.
>
> I'll upload a build with the correct 64-bit indexing and 32-bit BLAS and
> other Fortran libraries soon.  This configuration is the same as is typical for
> all other systems where the system BLAS, LAPACK, and other Fortran libraries are
> built with the default 32-bit INTEGER type.
>
> jwe
Sorry I have misled.  However, it is a good thing that my foolish performance was a chance
that you noticed your mistake.

I have re-read NEWS.
The option --enable-fortran-int64 enables to use "BLAS  LAPACK  QRUPDATE  ARPACK"

SuiteSparse internally used BLAS and LAPACK and --enable-fortran-int64 makes SuiteSparse handle large arrays?

How about glpk  Qhull?
Are patches for two libraries required to use large arrays in these libraries?

Tatsuro