benchmark.m version 1.3

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

benchmark.m version 1.3

Francesco Potorti`-9
I wrote yet another version of benchmark.m, thanks to the feedback I
received from some of you.  

Now it should be more accurate, because the time is measured in chunks
of around 3 seconds, so most overheads should become negligible on
every machine.  There are more tests and a general performance index
made from the geometric mean of the individual test indexes.  The
indexes are relative to a Sun Sparc 2.  The problems with the time
measurements should have gone away.  The total virtual memory needed
is around 10M, while the RSS tops at around 6M.

Evan Thomas pointed out that, while the Sparc2 may be a good reference
point because of its standard configuration and its relative slowness,
it will be increasingly difficult to find, so reference numbers could
be hard to get for future tests.  If anyone comes up with a better
refererence machine, we can move from Sparc2.

Here are the numbers we get for Alpha and 50MHz Intel.  Notice that
for Alpha I can usually get performance indices around 7, but I got
6.1 once, because the machine is always heavily loaded.  I'd like to
collect more benchmark results form you.  I will post them when I have
some.

Francesco Potorti` <[hidden email]>
  DEC Alpha 2100
Octave benchmark version bm 1.3
Speed of octave 1.1.1 on alpha-dec-osf1.3 relative to Sun Sparc 2
Matrix inversion (LAPACK)         9.24 +/- 4.2% (49 runs)
Schur decomposition (LAPACK)      8.41 +/- 1.3% (14 runs)
Differential equation (LSODE)     5.49 +/- 4.3% (14 runs)
Fourier transforms (FFTPACK)      6.79 +/- 2.4% (35 runs)
for loop                          6.43 +/- 0.9% (21 runs)
-- Performance index (bm 1.3): 7.1


Michael Smolsky <[hidden email]>
  intel 486dx2-50 20MBt ram, 256 cache linux box
Octave benchmark version bm 1.3
Speed of octave 1.1.1 on i486-unknown-linux relative to Sun Sparc 2
Matrix inversion (LAPACK)         0.74 +/- 13.3% (140 runs)
Schur decomposition (LAPACK)      0.67 +/- 14.5% (40 runs)
Differential equation (LSODE)     1.37 +/- 6.0% (40 runs)
Fourier transforms (FFTPACK)      0.48 +/- 2.2% (35 runs)
for loop                          1.53 +/- 8.9% (60 runs)
-- Performance index (bm 1.3): 0.87


Reply | Threaded
Open this post in threaded view
|

Re: benchmark.m version 1.3

Przemek Klosowski-4
This is a result of running octave on 75/90MHz R8000. Octave here is
highly 'pessimized' because I couldn't get reliable gcc under Irix
6.1, and therefore Octave is compiled under 32-bit Irix 5.3, and it
doesn't use the fast floating point instructions available on R8000.

Octave benchmark version bm 1.3
Speed of octave 1.1.1 on mips-sgi-irix5.3 relative to Sun Sparc 2
Matrix inversion (LAPACK)         4.44 +/- 13.9% (80 runs)
Schur decomposition (LAPACK)      3.09 +/- 0.0% (7 runs)
Differential equation (LSODE)     3.72 +/- 0.3% (7 runs)
Fourier transforms (FFTPACK)      3.48 +/- 0.4% (21 runs)
for loop                          4.14 +/- 1.2% (14 runs)
-- Performance index (bm 1.3): 3.7