Behold! Buildbot's waterfall display has never been so green

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Behold! Buildbot's waterfall display has never been so green

John W. Eaton
Administrator
See attached screenshot.  Or visit

   http://buildbot.octave.org:8010/waterfall

I hope we can keep things all green all the time.

Also, can someone please give the OS X build some love and make it green
as well?

jwe

buildbot.png (161K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Behold! Buildbot's waterfall display has never been so green

mmuetzel
The Windows cross builds could also need some love :)
Summary:

  PASS                            15807
  FAIL                                1
  REGRESSION                          4
  XFAIL (expected failure)            5
  XFAIL (reported bug)               34
  SKIPPED (feature)                  58
  SKIPPED (run-time condition)       26

Those 4 regressions are bug 45507. The fail is intermittent for
"__opengl_info__ ()".

fntests_fails.log
<http://octave.1599824.n4.nabble.com/file/t371856/fntests_fails.log>  



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

Reply | Threaded
Open this post in threaded view
|

Re: Behold! Buildbot's waterfall display has never been so green

John W. Eaton
Administrator
On 11/11/2017 02:58 PM, mmuetzel wrote:

> The Windows cross builds could also need some love :)
> Summary:
>
>    PASS                            15807
>    FAIL                                1
>    REGRESSION                          4
>    XFAIL (expected failure)            5
>    XFAIL (reported bug)               34
>    SKIPPED (feature)                  58
>    SKIPPED (run-time condition)       26
>
> Those 4 regressions are bug 45507. The fail is intermittent for
> "__opengl_info__ ()".
>
> fntests_fails.log
> <http://octave.1599824.n4.nabble.com/file/t371856/fntests_fails.log>

True.

I'd like to add testing for the Windows builds.  Since they are cross
compiled, we'll need to do something like this once the nsis installer
or zip file is built:

   * start a VM running windows that can accept shell commands over an
ssh network connection

   * transfer the nsis installer or zip file

   * run the installer or unpack the zip file

   * run the octave tests

This could be one or more buildbot steps.  I'm not exactly sure how to
make the first part work.  It would be nice if it could use virtualbox
since that's what I already use but other solutions would also be OK if
someone can show me what to do.

jwe


Reply | Threaded
Open this post in threaded view
|

Re: Behold! Buildbot's waterfall display has never been so green

Mike Miller-4
In reply to this post by John W. Eaton
On Sat, Nov 11, 2017 at 10:32:02 -0500, John W. Eaton wrote:
> Also, can someone please give the OS X build some love and make it green as
> well?

We've looked at some of these before, here is a short summary of the
test failures on macOS.

* LLVM C++ library defect in number parsing. This results in Octave
  unable to parse a string like "4i" into a complex number with
  functions like scanf and str2double.

  This bug occurs on any operating system when Octave is built with the
  LLVM clang compiler and the LLVM C++ standard library.

  The LLVM devs are aware of this and seem to be waiting for some C++
  standards work to clarify this case before making any changes.

  LLVM bug https://bugs.llvm.org/show_bug.cgi?id=17782

  Octave bug https://savannah.gnu.org/bugs/?47413 (closed won't fix)

  This causes test failures in dlmread.cc, str2double.cc, importdata.m,
  and io.tst.

* Less accurate inverse trigonometric and hyperbolic functions in the
  macOS system math library?

  I'm not sure if anyone has looked into these yet.

  This broad description is meant to capture test failures in mappers.cc
  and asech.m (functions acos, acosh, asin, asinh, and asech).

  Most or all of these may be due to system library accuracy problems,
  which I think we also may see on Windows in some of these cases.

* The default system temporary directory seems to be symlinked or
  containerized somehow on macOS. The system defines the default
  temporary directory to be "/var/tmp" (normally "/tmp" on GNU/Linux
  systems). In Octave's test for fileattrib, it tries to ensure that the
  "Name" property is the same, but "/private/var/tmp" is returned.

  This test in fileattrib.m could be dropped or modified to avoid making
  the assumption that the default temporary directory is not a symlink.

  Maybe a simple canonicalize_file_name will fix this.

* There is a test in camzoom.m that tries to assert a specific viewing
  angle, which is failing with a slight tolerance error of 2*eps. I
  assume this would be safe to fix with a tolerance argument.

* The last two sorting tests in complex.tst are failing when the single
  precision and double precision results are compared to each other. I
  don't know if anyone has looked into this or where the underlying
  error happens.

--
mike

signature.asc (849 bytes) Download Attachment