|
I am trying out some test driven development in Octave as a precursor to trying it elsewhere. However I have come slightly unstuck in trying to test for expected errors/failures.
For example I have started to write a function which requires two parameters and then can accept optional parameters, provided as a pair: getAccelData(required1, required2) getAccelData(required1, required2, optional1, optional2) optional1, if provided must be 'endAccel', and optional2 must be one of 'zero', last' or 'mean'. Following the help at http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I have got as far as: %!test % Check optional values - part one, expected failures %! error(getAccelData(tStep, vRamp, 'test', 'zero')); But on running my tests I get: !!!!! test failed No matching property found for: test Which is the expected error message! How do I write the test to pass i.e. successfully see the error message please? (Or alternatively is there a guide anywhere to TDD in Octave please that covers this? I've STFW but only seem to be able to find examples of testing for success, i.e. assert(...)) Thanks! Iain |
|
On 4 February 2013 12:29, IainIsm <[hidden email]> wrote:
> For example I have started to write a function which requires two parameters > and then can accept optional parameters, provided as a pair: > > getAccelData(required1, required2) > getAccelData(required1, required2, optional1, optional2) [snip] > !!!!! test failed > No matching property found for: test How are you writing getAccelData? Are you using inputParser? - Jordi G. H. _______________________________________________ Help-octave mailing list [hidden email] https://mailman.cae.wisc.edu/listinfo/help-octave |
|
On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave] <[hidden email]> wrote:
Hi - I am (was?) using a homebrew approach, but now I've looked up inputParser I might just use that instead (thank you)!
In the meantime, from the documentation at http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I was expecting that the following MWE would test successfully:
function output = mustBeZero(input) if (input ~= 0) error('Non-zero input!') end output=input; %!test fail(mustBeZero(1));
%!test assert(mustBeZero(0), 0); But what I get is: octave:29> test mustBeZero verbose >>>>> L:\octave\testing\mustBeZero.m
***** test fail(mustBeZero(1)); !!!!! test failed Non-zero input! But this doesn't match what I understood from the documentation since an 'expected failure' is occurring?
After a lot of playing the only way I can get the 'expected failure' test to pass is to use xtest: %!xtest fail(mustBeZero(1)); And I then get:
octave:30> test mustBeZero verbose >>>>> L:\octave\testing\mustBeZero.m ***** xtest fail(mustBeZero(1)); !!!!! known failure Non-zero input!
***** test assert(mustBeZero(0), 0); PASSES 2 out of 2 tests (1 expected failures) (Perhaps what I've found is a problem with the clarity of the documentation?) Iain |
|
On 5 February 2013 07:01, IainIsm <[hidden email]> wrote:
> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave] > <[hidden email]> wrote: >> >> On 4 February 2013 12:29, IainIsm <[hidden email]> wrote: >> > For example I have started to write a function which requires two >> > parameters and then can accept optional parameters, provided as a >> > pair: >> > >> > getAccelData(required1, required2) >> > getAccelData(required1, required2, optional1, optional2) >> [snip] >> > !!!!! test failed >> > No matching property found for: test >> >> How are you writing getAccelData? Are you using inputParser? > Hi - I am (was?) using a homebrew approach, but now I've looked up > inputParser I might just use that instead (thank you)! > > In the meantime, from the documentation at > http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I was > expecting that the following MWE would test successfully: The following what? > function output = mustBeZero(input) > if (input ~= 0) > error('Non-zero input!') > end > output=input; > > %!test fail(mustBeZero(1)); > %!test assert(mustBeZero(0), 0); > > But what I get is: > > octave:29> test mustBeZero verbose >>>>>> L:\octave\testing\mustBeZero.m > ***** test fail(mustBeZero(1)); > !!!!! test failed > Non-zero input! > > But this doesn't match what I understood from the documentation since an > 'expected failure' is occurring? You have to understand that the Octave parser is very, very dumb. In particular, under any circumstance, when you write foo(bar(1)) and foo and bar are functions, bar will get evaluated before foo no matter what foo is. There isn't a way for the foo function to create some sort of context in which bar(1) is evaluated specially (e.g. like a try-catch block). So in your test above, mustBeZero(1) is evaluated first, an error gets thrown, and control never reaches the fail() function. Instead, the fail() function expects a string, like so: fail("mustBeZero(1)") and internally uses evalin to evaluate this string in a special try-catch context: http://hg.savannah.gnu.org/hgweb/octave/file/59715612ea72/scripts/testfun/fail.m#l85 > After a lot of playing the only way I can get the 'expected failure' test to > pass is to use xtest: > > %!xtest fail(mustBeZero(1)); Unlike fail, the test and xtest do evaluate their code in a special context. In effect, the %! blocks are not evaluated by the interpreter directly, but by specialised code that later passes on the code to the Octave interpreter, similar to evalin inside a try-catch block. Therefore, and %!xtest block automatically stringifies its contents before evaluating, so it can catch the mustBeZero(1) error. This is therefore equivalent to your test above: %!xtest mustBeZero(1) > (Perhaps what I've found is a problem with the clarity of the > documentation?) Suggest alternative wording, and I will incorporate it into the manual. - Jordi G. H. _______________________________________________ Help-octave mailing list [hidden email] https://mailman.cae.wisc.edu/listinfo/help-octave |
|
On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave]
<[hidden email]> wrote: > On 5 February 2013 07:01, IainIsm <[hidden email]> wrote: > >> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave] [snip] >> In the meantime, from the documentation at >> http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I >> was >> expecting that the following MWE would test successfully: > > The following what? Sorry, minimum working example (shortest bit of code that I could create that would exhibit the behaviour I was seeing). > Instead, the fail() function expects a string, like so: > > fail("mustBeZero(1)") Ah-ha! Thank you! I hadn't understood the significance of "Note that code is a string" in the documentation. >> (Perhaps what I've found is a problem with the clarity of the >> documentation?) > > Suggest alternative wording, and I will incorporate it into the > manual. Between: ***** If you want to temporarily disable a test block, put # in place of the block type. This creates a comment block which is echoed in the log file but not executed. For example: %!#demo %! t = [0:0.01:2*pi]; x = sin (t); %! plot (t, x); %! # you should now see a sine wave in your figure window" ***** and: ***** "Block type summary:" ***** I would insert: ***** The following trivial code snippet provides examples for the use of fail, assert, error and xtest: function output = mustBeZero(input) if (input ~= 0) error('Non-zero input!') end output = input; %!fail('mustBeZero(1)'); %!assert(mustBeZero(0), 0); %!error <Non-zero> mustBeZero(1); %!xtest error('This code generates an error'); Output in response to: test mustBeZero verbose Is: >>>>><path to>\mustBeZero.m ***** fail('mustBeZero(1)'); ***** assert(mustBeZero(0), 0); ***** error <Non-zero> mustBeZero(1); ***** xtest fail(error('This code generates an error')); !!!!! known failure This generates an error PASSES 4 out of 4 tests (1 expected failures) ***** And then in the section for fail: Change: ***** — Function File: fail (code) — Function File: fail (code, pattern) — Function File: fail (code, 'warning', pattern) Return true if code fails with an error message matching pattern, otherwise produce an error. Note that code is a string and if code runs successfully, the error produced is: expected error but got none If the code fails with a different error, the message produced is: ***** To: ***** — Function File: fail (code) — Function File: fail (code, pattern) — Function File: fail (code, 'warning', pattern) If called with one argument, return true if code fails. If code runs successfully, the error produced is: expected error <.> but got none NOTE: code must be in the form of a string that may be passed by fail to the Octave interpreter via the evalin function (i.e. a (quoted) string constant or a string variable). If called with two arguments, the behavior is similar to fail(code), except the return value will only be true if code fails with an error message containing pattern (case sensitive). If the code fails with a different error to that given in pattern, the message produced is: ***** Thoughts? On a related note, the documentation for evalin contains the text: "One example is the fail (‘code’, ‘pattern’) function which evaluates ‘code’ in the caller's context and checks that the error message it produces matches the given pattern": http://www.gnu.org/software/octave/doc/interpreter/Evaluation-in-a-Different-Context.html#doc%2devalin So I'm not totally sure what the convention for showing strings in the function documentation is (should be?) to make my suggestion above consistent with the remainder of the documentation. Thanks for your help, Iain |
|
In reply to this post by Jordi Gutiérrez Hermoso-2
On 5 February 2013 16:35, Iain Cunningham wrote:
> > On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave] > <[hidden email]> wrote: > > On 5 February 2013 07:01, IainIsm <[hidden email]> wrote: > > > >> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave] > [snip] [snip] > Thoughts? Did you get a chance to consider the proposal at all please Jordi? Iain |
|
On 26 February 2013 11:15, IainIsm <[hidden email]> wrote:
> On 5 February 2013 16:35, Iain Cunningham wrote: >> >> On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave] >> <[hidden email]> wrote: >> > On 5 February 2013 07:01, IainIsm <[hidden email]> wrote: >> > >> >> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave] >> [snip] > [snip] >> Thoughts? > > Did you get a chance to consider the proposal at all please Jordi? Sorry for the delay. I have put your suggestions on the stable branch: http://hg.savannah.gnu.org/hgweb/octave/rev/7fe4ea72ba4dw Thanks for the contribution. - Jordi G. H. _______________________________________________ Help-octave mailing list [hidden email] https://mailman.cae.wisc.edu/listinfo/help-octave |
|
On 4 March 2013 16:53, Jordi Gutiérrez Hermoso <[hidden email]> wrote:
> On 26 February 2013 11:15, IainIsm <[hidden email]> wrote: >> On 5 February 2013 16:35, Iain Cunningham wrote: >>> >>> On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave] >>> <[hidden email]> wrote: >>> > On 5 February 2013 07:01, IainIsm <[hidden email]> wrote: >>> > >>> >> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave] >>> [snip] >> [snip] >>> Thoughts? >> >> Did you get a chance to consider the proposal at all please Jordi? > > Sorry for the delay. I have put your suggestions on the stable branch: > > http://hg.savannah.gnu.org/hgweb/octave/rev/7fe4ea72ba4dw Sorry again, bad URL: http://hg.savannah.gnu.org/hgweb/octave/rev/7fe4ea72ba4d - Jordi G. H. _______________________________________________ Help-octave mailing list [hidden email] https://mailman.cae.wisc.edu/listinfo/help-octave |
| Powered by Nabble | Edit this page |
