Hardware to run Octave

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Hardware to run Octave

Jose Ramom Flores das Seixas
Hi

As part of a research project, I wrote a program that takes about 17
hours to produce the result, using only one of the processor cores. I
tried to use the parallel package, but without success.

Since the amount of time it takes to see the results is so long, the
process of checking and correcting errors is very slow, not to mention
that in my office the power goes out every now and then, wasting hours
spent on calculations.

My current computer will soon be 10 years old, and I was thinking of
buying a new one. Its CPU is an Intel i52500K @ 3.30 GHz with 4 cores
and 4 threads.

One of the goals I have for the new PC is for it to run Octave nicely. 
Previously processors increased their power by, among other things,
increasing the processor speed. But for a number of years now, the speed
of new processors has not increased, and other methods have been used to
improve them, such as using more cores.

So I'd like to read your opinions on which hardware would be more
suitable for running octave.

Surely yours

Ramom



Reply | Threaded
Open this post in threaded view
|

Re: Hardware to run Octave

Etienne Grossmann-7

  Hi Ramon,

sorry, I don't have much advice to offer on a new device, I use old hardware too, but don't have bulky tasks for Octave.

  About the parallel package. I never had luck either until today when I saw actual examples here: http://wiki.octave.org/Parallel_package. Did you try these examples? They kind of work for me (w/ Octave 6.1): 

>> function y = my_func(x), y = 42 * x; end      # Try with command-line-defined function
>> pararrayfun (4, @my_func, 1:8)
>> my_func(3)
ans = 126
>> pararrayfun (4, @(x) my_func(x), 1:8)          # Why no output?
                                                  # Try with file-defined function
>> system('echo "function y = my_func2(x), y = 42 * x; end" > my_func2.m')
ans = 0
>> addpath('.')
>> pararrayfun (4, @(x) my_func2(x), 1:8)         # Still no output?
>> my_func2(4)
ans = 168
>> arrayfun (@(x) my_func2(x), 1:8)               # This works but is not parallel
ans =

    42    84   126   168   210   252   294   336

>> pararrayfun (4, @(x) my_func2(x), 1:8)
>> z = pararrayfun (4, @(x) my_func2(x), 1:8)          # Aha! A return value is needed?
z =

    42    84   126   168   210   252   294   336

>> z = pararrayfun (4, @(x) my_func(x), 1:8)      # That's not the end of the story
z =

  -1  -1  -1  -1  -1  -1  -1  -1
 
>> z = pararrayfun (4, @my_func, 1:8)             # CL-defined my_func() is jinxed?
z =

  -1  -1  -1  -1  -1  -1  -1  -1

>> z = pararrayfun (4, @my_func2, 1:8)            # But file-defined my_func2() seems OK
z =

    42    84   126   168   210   252   294   336
                                                  # Let's check parallelism... LGTM
>> system('echo ''function y = my_delay_func(x), y = 42 * x; printf("Starting %i\\n", x); pause(2); printf ("--Ending %i\\n", x); end'' > my_delay_func.m')
ans = 0
>> system ('cat my_delay_func.m')
function y = my_delay_func(x), y = 42 * x; printf("Starting %i\n", x); pause(2); printf ("--Ending %i\n", x); end
ans = 0
>> my_delay_func(3)
Starting 3
--Ending 3
ans = 126
>> z = pararrayfun (4, @my_delay_func, 1:8)
Starting 1
Starting 4
Starting 3
Starting 2
--Ending 1
Starting 5
--Ending 4
Starting 6
--Ending 3
--Ending 2
Starting 7
Starting 8
--Ending 5
--Ending 6
--Ending 7
--Ending 8
z =

    42    84   126   168   210   252   294   336

   Does this not work for you? If not, what output do you get w/ the above command? What version of Octave are you using?

    HTH,

    Etienne



On Thu, Feb 18, 2021 at 7:47 AM Jose Ramom Flores das Seixas <[hidden email]> wrote:
Hi

As part of a research project, I wrote a program that takes about 17
hours to produce the result, using only one of the processor cores. I
tried to use the parallel package, but without success.

Since the amount of time it takes to see the results is so long, the
process of checking and correcting errors is very slow, not to mention
that in my office the power goes out every now and then, wasting hours
spent on calculations.

My current computer will soon be 10 years old, and I was thinking of
buying a new one. Its CPU is an Intel i52500K @ 3.30 GHz with 4 cores
and 4 threads.

One of the goals I have for the new PC is for it to run Octave nicely. 
Previously processors increased their power by, among other things,
increasing the processor speed. But for a number of years now, the speed
of new processors has not increased, and other methods have been used to
improve them, such as using more cores.

So I'd like to read your opinions on which hardware would be more
suitable for running octave.

Surely yours

Ramom





Reply | Threaded
Open this post in threaded view
|

Re: Hardware to run Octave

Jose Ramom Flores das Seixas
Às 06:38 de 20/02/21, Etienne Grossmann escreveu:

  Hi Ramon,

sorry, I don't have much advice to offer on a new device, I use old hardware too, but don't have bulky tasks for Octave.

  About the parallel package. I never had luck either until today when I saw actual examples here: http://wiki.octave.org/Parallel_package. Did you try these examples? They kind of work for me (w/ Octave 6.1): 

>> function y = my_func(x), y = 42 * x; end      # Try with command-line-defined function
>> pararrayfun (4, @my_func, 1:8)
>> my_func(3)
ans = 126
>> pararrayfun (4, @(x) my_func(x), 1:8)          # Why no output?
                                                  # Try with file-defined function
>> system('echo "function y = my_func2(x), y = 42 * x; end" > my_func2.m')
ans = 0
>> addpath('.')
>> pararrayfun (4, @(x) my_func2(x), 1:8)         # Still no output?
>> my_func2(4)
ans = 168
>> arrayfun (@(x) my_func2(x), 1:8)               # This works but is not parallel
ans =

    42    84   126   168   210   252   294   336

>> pararrayfun (4, @(x) my_func2(x), 1:8)
>> z = pararrayfun (4, @(x) my_func2(x), 1:8)          # Aha! A return value is needed?
z =

    42    84   126   168   210   252   294   336

>> z = pararrayfun (4, @(x) my_func(x), 1:8)      # That's not the end of the story
z =

  -1  -1  -1  -1  -1  -1  -1  -1
 
>> z = pararrayfun (4, @my_func, 1:8)             # CL-defined my_func() is jinxed?
z =

  -1  -1  -1  -1  -1  -1  -1  -1

>> z = pararrayfun (4, @my_func2, 1:8)            # But file-defined my_func2() seems OK
z =

    42    84   126   168   210   252   294   336
                                                  # Let's check parallelism... LGTM
>> system('echo ''function y = my_delay_func(x), y = 42 * x; printf("Starting %i\\n", x); pause(2); printf ("--Ending %i\\n", x); end'' > my_delay_func.m')
ans = 0
>> system ('cat my_delay_func.m')
function y = my_delay_func(x), y = 42 * x; printf("Starting %i\n", x); pause(2); printf ("--Ending %i\n", x); end
ans = 0
>> my_delay_func(3)
Starting 3
--Ending 3
ans = 126
>> z = pararrayfun (4, @my_delay_func, 1:8)
Starting 1
Starting 4
Starting 3
Starting 2
--Ending 1
Starting 5
--Ending 4
Starting 6
--Ending 3
--Ending 2
Starting 7
Starting 8
--Ending 5
--Ending 6
--Ending 7
--Ending 8
z =

    42    84   126   168   210   252   294   336

   Does this not work for you? If not, what output do you get w/ the above command? What version of Octave are you using?

    HTH,

    Etienne



On Thu, Feb 18, 2021 at 7:47 AM Jose Ramom Flores das Seixas <[hidden email]> wrote:
Hi

As part of a research project, I wrote a program that takes about 17
hours to produce the result, using only one of the processor cores. I
tried to use the parallel package, but without success.

Since the amount of time it takes to see the results is so long, the
process of checking and correcting errors is very slow, not to mention
that in my office the power goes out every now and then, wasting hours
spent on calculations.

My current computer will soon be 10 years old, and I was thinking of
buying a new one. Its CPU is an Intel i52500K @ 3.30 GHz with 4 cores
and 4 threads.

One of the goals I have for the new PC is for it to run Octave nicely. 
Previously processors increased their power by, among other things,
increasing the processor speed. But for a number of years now, the speed
of new processors has not increased, and other methods have been used to
improve them, such as using more cores.

So I'd like to read your opinions on which hardware would be more
suitable for running octave.

Surely yours

Ramom



Thanks for the tips.

I was able to reproduce the examples, and even made new attempts successfully. But I have not yet been able to apply the pararrayfun command to my program, since the function to be parallelized is quite complex. I'm still trying it.

Surely yours
Ramom





Reply | Threaded
Open this post in threaded view
|

Re: Hardware to run Octave

vrozos
In reply to this post by Jose Ramom Flores das Seixas
Your current machine configuration, though 10 year-old, is not that bad. Do
not expect any significant improvement with a new one. You have three
options:

1. find a more efficient algorithm for your problem,
2. if you have nested for-loops you cannot avoid, consider implementing them
in C -> mex
3. parallelization (alas, multi-core processing is not a easy thing).

BTW, if your problem involves eigenvalues, eigen-3.3.5 library does
miracles. If you need normxcorr, I can suggest a nice solution too.



--
Sent from: https://octave.1599824.n4.nabble.com/Octave-General-f1599825.html


Reply | Threaded
Open this post in threaded view
|

Re: Hardware to run Octave

Jose Ramom Flores das Seixas
Às 20:51 de 23/02/21, vrozos wrote:

> Your current machine configuration, though 10 year-old, is not that bad. Do
> not expect any significant improvement with a new one. You have three
> options:
>
> 1. find a more efficient algorithm for your problem,
> 2. if you have nested for-loops you cannot avoid, consider implementing them
> in C -> mex
> 3. parallelization (alas, multi-core processing is not a easy thing).
>
> BTW, if your problem involves eigenvalues, eigen-3.3.5 library does
> miracles. If you need normxcorr, I can suggest a nice solution too.
Thanks to Etienne Grossmann's tips, and after some time spent
investigating the use of parrarrayfun, I was able to "parallelize" my
algorithm a little more and use all 4 cores of my computer, dividing the
calculation time by 2.8, so that now the program takes about 6 hours to
finish.

In order to help people who may have the same difficulty in
understanding the spartan documentation of the parallel package, I am
attaching a file with two relatively simple examples showing how to use
the parrarrayfun function with one and two dimensional arrays.

Surely yours

Ramom




Examples of how to use parrarrayfun.pdf (138K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hardware to run Octave

siko1056
On 3/2/21 5:06 AM, Jose Ramom Flores das Seixas wrote:

>
> In order to help people who may have the same difficulty in
> understanding the spartan documentation of the parallel package, I am
> attaching a file with two relatively simple examples showing how to use
> the parrarrayfun function with one and two dimensional arrays.
>
> Surely yours
>
> Ramom
>

Thanks for this file.  I added it to the wiki [1] to hopefully increase
the audience.

Kai

[1] https://wiki.octave.org/Parallel_package