Re: Behold! Buildbot's waterfall display has never been so green

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Behold! Buildbot's waterfall display has never been so green

Rik-4
On Sat, Nov 11, 2017 at 10:32:02 -0500, John W. Eaton wrote:
> Also, can someone please give the OS X build some love and make it green as
> well?

I did that. See bug https://savannah.gnu.org/bugs/?52627. All of the failing tests have been either fixed, or understood to be outside of Octave's purview and marked accordingly.  The latest build log is available at http://buildbot.octave.org:8010/builders/clang-osx/builds/832/steps/test/logs/stdio which shows

  PASS                            15898
  FAIL                                0
  REGRESSION                          5
  XFAIL (reported bug)               45
  XFAIL (expected failure)            6
  SKIP (missing feature)            154
  SKIP (run-time condition)          23

The test still shows red as failing because of warnings, but I think this is actually incorrect and something to do with regular expressions and how pass/fail is determined.  The warnings are

 saved_opts = warning ();
 warning ("off", "all");
 warning (saved_opts);
 warning ("off", "ode15s:mass_state_dependent_provided", "local");
 warning ("off", "ode15s:mass_state_dependent_provided", "local");
 warning ("off", "ode15s:mass_state_dependent_provided", "local");
 warning ("off", "ode15s:mass_state_dependent_provided", "local");
These are not actually warnings delivered to stderr, they are part of the code that was being tested which happend to contain the word "warning".

--Rik
Reply | Threaded
Open this post in threaded view
|

Re: Behold! Buildbot's waterfall display has never been so green

John W. Eaton
Administrator
On 01/06/2018 10:10 AM, Rik wrote:
> On Sat, Nov 11, 2017 at 10:32:02 -0500, John W. Eaton wrote:
>>/Also, can someone please give the OS X build some love and make it green as/
>>/well?/
>
> I did that. See bug https://savannah.gnu.org/bugs/?52627. All of the
> failing tests have been either fixed, or understood to be outside of
> Octave's purview and marked accordingly.

Thanks for the fixes and for checking.

> The latest build log is
> available at
> http://buildbot.octave.org:8010/builders/clang-osx/builds/832/steps/test/logs/stdio 
> which shows
>
>    PASS                            15898
>    FAIL                                0
>    REGRESSION                          5
>    XFAIL (reported bug)               45
>    XFAIL (expected failure)            6
>    SKIP (missing feature)            154
>    SKIP (run-time condition)          23
>
> The test still shows red as failing because of warnings, but I think
> this is actually incorrect and something to do with regular expressions
> and how pass/fail is determined.

I think buildbot is flagging it as failed because of the 5 failures that
are reported as regressions.  So these are problems that have bug
reports and have been tagged as fixed in the tests.

> The warnings are
>
>   saved_opts = warning ();
>   warning ("off", "all");
>   warning (saved_opts);
>   warning ("off", "ode15s:mass_state_dependent_provided", "local");
>   warning ("off", "ode15s:mass_state_dependent_provided", "local");
>   warning ("off", "ode15s:mass_state_dependent_provided", "local");
>   warning ("off", "ode15s:mass_state_dependent_provided", "local");
>
> These are not actually warnings delivered to stderr, they are part of
> the code that was being tested which happend to contain the word "warning".

Yeah, I'm not sure how to configure buildbot to filter those out.

jwe