Hardening BIST tests

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Hardening BIST tests

Rik-4
All,

I took the first steps toward hardening the BIST tests against local
preference files.  I ran the test suite after using save_default_options
("-ascii") and corrected nine problems in this cset
(https://hg.savannah.gnu.org/hgweb/octave/rev/63b5a17ba30f).

Running the test suite with default graphics toolkit of "fltk" or "gnuplot"
went fine and revealed no problems.

I could not run the test suite in a read-only directory as I immediately got

__run_test_suite__: could not open /home/rik/..../fntests.log for writing

--Rik


Reply | Threaded
Open this post in threaded view
|

Re: Hardening BIST tests

Mike Miller-4
On Fri, Feb 08, 2019 at 16:18:55 -0800, Rik wrote:
> I could not run the test suite in a read-only directory as I immediately got
>
> __run_test_suite__: could not open /home/rik/..../fntests.log for writing

This is pretty easy to work around

    > test/fntests.log
    chmod -R a-w . ${srcdir}   ## in my case, srcdir == ..
    chmod u+w test/fntests.log
    make check

No errors here. I've done this before and found only this bug, which is
still open, but I had added a FIXME workaround a few weeks ago:

  https://savannah.gnu.org/bugs/?53578

Btw, running the test suite read-only is done very often. Any time a
non-privileged user starts Octave installed as root (or administrator on
Windows) and runs '__run_test_suite__'. It's also run about once a day
on the Debian CI infrastructure.

More hardening ideas, some of which I have tested already:

 - run with DISPLAY set to empty, or with --no-window-system
 - run with HOME set to / or to a nonexistent directory
 - run with TMPDIR set to /var/tmp
 - run with "format long" or some other custom format (e.g. bug #55539)

--
mike

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hardening BIST tests

Rik-4
In reply to this post by Rik-4
On 02/08/2019 06:03 PM, [hidden email] wrote:
Subject:
Re: Hardening BIST tests
From:
Mike Miller [hidden email]
Date:
02/08/2019 06:03 PM
To:
[hidden email]
List-Post:
[hidden email]
Precedence:
list
MIME-Version:
1.0
References:
<MTAwMDAwMC5ub21hZA.1549671565@quikprotect>
In-Reply-To:
<MTAwMDAwMC5ub21hZA.1549671565@quikprotect>
Message-ID:
[hidden email]
Content-Type:
multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ZGiS0Q5IWpPtfppv"
Message:
5

More hardening ideas, some of which I have tested already:

 - run with DISPLAY set to empty, or with --no-window-system
 - run with HOME set to / or to a nonexistent directory
 - run with TMPDIR set to /var/tmp
 - run with "format long" or some other custom format (e.g. bug #55539)

--

I ran three more scenarios

 - save_precision (1) : 1 problem fixed here (https://hg.savannah.gnu.org/hgweb/octave/rev/87d3baf1c024)
 - format hex : 10 problems fixed here (https://hg.savannah.gnu.org/hgweb/octave/rev/daa281e03d9a)
 - save_default_options ("-text -z") with a version of Octave without zlib support : 26 failures, but I figure that was okay.  The scenario where this would occur is unlikely.  A developer with the '-z' option is switching back and forth between multiple installed versions some with / some without ZLIB.

Maybe this happens all the time, but I think Windows user's should try __run_test_suite__ when they are on a different drive letter then the location where Octave was installed.

--Rik

Reply | Threaded
Open this post in threaded view
|

Re: Hardening BIST tests

Philip Nienhuis
Rik-4 wrote
> On 02/08/2019 06:03 PM,

> octave-maintainers-request@

>  wrote:
>> Subject:
>> Re: Hardening BIST tests
>> From:
>> Mike Miller &lt;

> mtmiller@

> &gt;
>> Date:
>> 02/08/2019 06:03 PM
>>
>> To:
>>

> octave-maintainers@

>>
>> List-Post:
>> &lt;mailto:

> octave-maintainers@

> &gt;
>> Precedence:
>> list
>> MIME-Version:
>> 1.0
>> References:
>> &lt;MTAwMDAwMC5ub21hZA.1549671565@quikprotect&gt;
>> In-Reply-To:
>> &lt;MTAwMDAwMC5ub21hZA.1549671565@quikprotect&gt;
>> Message-ID:
>> <

> 20190209020301.GA25139@.mtmxr

>>
>> Content-Type:
>> multipart/signed; micalg=pgp-sha256;
>> protocol="application/pgp-signature"; boundary="ZGiS0Q5IWpPtfppv"
>> Message:
>> 5
>>
>>
>> More hardening ideas, some of which I have tested already:
>>
>>  - run with DISPLAY set to empty, or with --no-window-system
>>  - run with HOME set to / or to a nonexistent directory
>>  - run with TMPDIR set to /var/tmp
>>  - run with "format long" or some other custom format (e.g. bug #55539)
>>
>> --
>
> I ran three more scenarios
>
>  - save_precision (1) : 1 problem fixed here
> (https://hg.savannah.gnu.org/hgweb/octave/rev/87d3baf1c024)
>  - format hex : 10 problems fixed here
> (https://hg.savannah.gnu.org/hgweb/octave/rev/daa281e03d9a)
>  - save_default_options ("-text -z") with a version of Octave without zlib
> support : 26 failures, but I figure that was okay.  The scenario where
> this
> would occur is unlikely.  A developer with the '-z' option is switching
> back and forth between multiple installed versions some with / some
> without
> ZLIB.
>
> Maybe this happens all the time, but I think Windows user's should try
> __run_test_suite__ when they are on a different drive letter then the
> location where Octave was installed.

Windows installations of Octave are a somewhat self-contained thing due to
the way MSYS2 is set up, maybe that interferes a bit.
To invoke __run_test_suite__.m, cd'ing to some temp subdir should do. An
easy way to do this would be

cd (getenv ("TMP"))

which will get you into C:\Users\<username>\Appdata\Local\Temp where you'll
have guaranteed r/w permission - given sufficient space on the c: drive.
(environment variables TEMP and TMP (both pointing to the same location) are
AFAIK set by Windows automatically at boot time.)

Philip




--
Sent from: http://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html