Re: boosting tolerances on gammainc BIST tests

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: boosting tolerances on gammainc BIST tests

Rik-4
On 02/05/2019 02:13 AM, [hidden email] wrote:
Subject:
Re: Tested 5.0.91
From:
Mike Miller [hidden email]
Date:
02/04/2019 08:32 PM
To:
Christian Himpe [hidden email]
CC:
Colin Macdonald [hidden email], [hidden email]
List-Post:
[hidden email]
Precedence:
list
MIME-Version:
1.0
References:
<20190205015620.5b320700@XYLAZ>
In-Reply-To:
<20190205015620.5b320700@XYLAZ>
Message-ID:
[hidden email]
Content-Type:
multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="BXVAT5kNtrzKuDFl"
Message:
5

Hi Christian,

On Tue, Feb 05, 2019 at 01:56:20 +0100, Christian Himpe wrote:
I compiled octave-5.0.91 on Ubuntu 18.04 with the self-compiled
library stack:

* OpenBLAS 0.3.5
* FlexiBLAS 2.0.0
* qrupdate 1.1.2
* SuiteSparse 5.4.0
* arpack-ng 3.7.0
* sundials 2.7.0
When you run the test suite are you using FlexiBLAS with OpenBLAS
selected? I assume so.

Is your Ubuntu system x86-64?

whereas the fails are in:

* plot/appearance/camlookat.m (1 FAIL)
* specfun/gammainc.m (3 FAILs)
In your fntests.log file, all 4 failures are because the difference is
slightly out of the test tolerance.

I think the gammainc.m test tolerances were chosen empirically, and they
have been increased in the past (https://savannah.gnu.org/bugs/?53437).

@Mike: No objection from me to relaxing the tolerances if the miss is only a further few eps.  If it is larger, maybe 10 eps, then it probably deserves a closer look.
Cc'ing Colin to comment on the camlookat.m test results and tolerances:

!!!!! test failed
ASSERT errors for:  assert (dir,dir2,-2 * eps)

  Location  |  Observed  |  Expected  |  Reason
    (2)        0.72803      0.72803      Rel err 4.5749e-16 exceeds tol 4.4409e-16 by 1e-17

@Christian: What OpenGL driver are you using?  Is this software rendering or a dedicated card?

If you switch back to OpenBLAS from FlexiBLAS do the errors go away?

Is the compiler gcc?  version?  Did you use any other configure options like '--disable-float-truncate', or was this a vanilla configure and build?

--Rik
Reply | Threaded
Open this post in threaded view
|

Re: boosting tolerances on gammainc BIST tests

Mike Miller-4
On Tue, Feb 05, 2019 at 09:46:25 -0800, Rik wrote:
> @Mike: No objection from me to relaxing the tolerances if the miss is only
> a further few eps.  If it is larger, maybe 10 eps, then it probably
> deserves a closer look.

The gammainc.m test failures on Christian's system

     ()         0.4906       0.4906      Rel err 1.1315e-15 exceeds tol 1.1102e-15 by 2e-17
     ()        17.3984      17.3984      Rel err 6.1259e-16 exceeds tol 4.4409e-16 by 2e-16
     ()        18.0654      18.0654      Rel err 1.5733e-15 exceeds tol 1.5543e-15 by 2e-17

are from these three tests

%!assert (gammainc (200, 200, "upper"), 0.49059658199276367497217454, -5*eps)
%!assert (gammainc(1000, 1000.5, "scaledlower"), 39.48467539583672271, -2*eps)
%!assert (gammainc (200, 200, "scaledlower"), 18.065406676779221643065, -7*eps)

and look like they will all be fixed by increasing by 1 eps, I'll make
that change on stable soon.

> If you switch back to OpenBLAS from FlexiBLAS do the errors go away?

It should be a pass through, I used the same combination of FlexiBLAS
2.0.0 and OpenBLAS 0.3.5 and was not able to reproduce any of the fails.

--
mike

signature.asc (849 bytes) Download Attachment