Quantcast

TDD in Octave, guidance/tutorial?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

TDD in Octave, guidance/tutorial?

IainIsm
I am trying out some test driven development in Octave as a precursor to trying it elsewhere.  However I have come slightly unstuck in trying to test for expected errors/failures.

For example I have started to write a function which requires two parameters and then can accept optional parameters, provided as a pair:

getAccelData(required1, required2)
getAccelData(required1, required2, optional1, optional2)

optional1, if provided must be 'endAccel', and optional2 must be one of 'zero', last' or 'mean'.

Following the help at http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I have got as far as:

%!test % Check optional values - part one, expected failures
%!  error(getAccelData(tStep, vRamp, 'test', 'zero'));

But on running my tests I get:

!!!!! test failed
No matching property found for: test

Which is the expected error message!

How do I write the test to pass i.e. successfully see the error message please?  (Or alternatively is there a guide anywhere to TDD in Octave please that covers this?  I've STFW but only seem to be able to find examples of testing for success, i.e. assert(...))

Thanks!

Iain

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: TDD in Octave, guidance/tutorial?

Jordi Gutiérrez Hermoso-2
On 4 February 2013 12:29, IainIsm <[hidden email]> wrote:
> For example I have started to write a function which requires two parameters
> and then can accept optional parameters, provided as a pair:
>
> getAccelData(required1, required2)
> getAccelData(required1, required2, optional1, optional2)
[snip]
> !!!!! test failed
> No matching property found for: test

How are you writing getAccelData? Are you using inputParser?

- Jordi G. H.
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: TDD in Octave, guidance/tutorial?

IainIsm
On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave] <[hidden email]> wrote:
On 4 February 2013 12:29, IainIsm <[hidden email]> wrote:
> For example I have started to write a function which requires two parameters
> and then can accept optional parameters, provided as a pair:
>
> getAccelData(required1, required2)
> getAccelData(required1, required2, optional1, optional2)
[snip]
> !!!!! test failed
> No matching property found for: test

How are you writing getAccelData? Are you using inputParser?

- Jordi G. H.
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave

Hi - I am (was?) using a homebrew approach, but now I've looked up inputParser I might just use that instead (thank you)!

In the meantime, from the documentation at http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I was expecting that the following MWE would test successfully:

function output = mustBeZero(input)
if (input ~= 0)
  error('Non-zero input!')
end
output=input;

%!test fail(mustBeZero(1));
%!test assert(mustBeZero(0), 0);

But what I get is:

octave:29> test mustBeZero verbose
>>>>> L:\octave\testing\mustBeZero.m
  ***** test fail(mustBeZero(1));
!!!!! test failed
Non-zero input!

But this doesn't match what I understood from the documentation since an 'expected failure' is occurring?

After a lot of playing the only way I can get the 'expected failure' test to pass is to use xtest:

%!xtest fail(mustBeZero(1));

And I then get:
octave:30> test mustBeZero verbose
>>>>> L:\octave\testing\mustBeZero.m
  ***** xtest fail(mustBeZero(1));
!!!!! known failure
Non-zero input!
  ***** test assert(mustBeZero(0), 0);
PASSES 2 out of 2 tests (1 expected failures)

(Perhaps what I've found is a problem with the clarity of the documentation?)

Iain

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: TDD in Octave, guidance/tutorial?

Jordi Gutiérrez Hermoso-2
On 5 February 2013 07:01, IainIsm <[hidden email]> wrote:

> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave]
> <[hidden email]> wrote:
>>
>> On 4 February 2013 12:29, IainIsm <[hidden email]> wrote:
>> > For example I have started to write a function which requires two
>> > parameters and then can accept optional parameters, provided as a
>> > pair:
>> >
>> > getAccelData(required1, required2)
>> > getAccelData(required1, required2, optional1, optional2)
>> [snip]
>> > !!!!! test failed
>> > No matching property found for: test
>>
>> How are you writing getAccelData? Are you using inputParser?

> Hi - I am (was?) using a homebrew approach, but now I've looked up
> inputParser I might just use that instead (thank you)!
>
> In the meantime, from the documentation at
> http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I was
> expecting that the following MWE would test successfully:

The following what?

> function output = mustBeZero(input)
> if (input ~= 0)
>   error('Non-zero input!')
> end
> output=input;
>
> %!test fail(mustBeZero(1));
> %!test assert(mustBeZero(0), 0);
>
> But what I get is:
>
> octave:29> test mustBeZero verbose
>>>>>> L:\octave\testing\mustBeZero.m
>   ***** test fail(mustBeZero(1));
> !!!!! test failed
> Non-zero input!
>
> But this doesn't match what I understood from the documentation since an
> 'expected failure' is occurring?

You have to understand that the Octave parser is very, very dumb.

In particular, under any circumstance, when you write

   foo(bar(1))

and foo and bar are functions, bar will get evaluated before foo no
matter what foo is. There isn't a way for the foo function to create
some sort of context in which bar(1) is evaluated specially (e.g. like
a try-catch block). So in your test above, mustBeZero(1) is evaluated
first, an error gets thrown, and control never reaches the fail()
function.

Instead, the fail() function expects a string, like so:

    fail("mustBeZero(1)")

and internally uses evalin to evaluate this string in a special
try-catch context:

    http://hg.savannah.gnu.org/hgweb/octave/file/59715612ea72/scripts/testfun/fail.m#l85

> After a lot of playing the only way I can get the 'expected failure' test to
> pass is to use xtest:
>
> %!xtest fail(mustBeZero(1));

Unlike fail, the test and xtest do evaluate their code in a special
context. In effect, the %! blocks are not evaluated by the interpreter
directly, but by specialised code that later passes on the code to the
Octave interpreter, similar to evalin inside a try-catch block.
Therefore, and %!xtest block automatically stringifies its contents
before evaluating, so it can catch the mustBeZero(1) error. This is
therefore equivalent to your test above:

    %!xtest mustBeZero(1)

> (Perhaps what I've found is a problem with the clarity of the
> documentation?)

Suggest alternative wording, and I will incorporate it into the
manual.

- Jordi G. H.
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: TDD in Octave, guidance/tutorial?

IainIsm
On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave]
<[hidden email]> wrote:
> On 5 February 2013 07:01, IainIsm <[hidden email]> wrote:
>
>> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave]
[snip]
>> In the meantime, from the documentation at
>> http://www.gnu.org/software/octave/doc/interpreter/Test-Functions.html I
>> was
>> expecting that the following MWE would test successfully:
>
> The following what?

Sorry, minimum working example (shortest bit of code that I could
create that would exhibit the behaviour I was seeing).

> Instead, the fail() function expects a string, like so:
>
>     fail("mustBeZero(1)")

Ah-ha!  Thank you!  I hadn't understood the significance of "Note that
code is a string" in the documentation.

>> (Perhaps what I've found is a problem with the clarity of the
>> documentation?)
>
> Suggest alternative wording, and I will incorporate it into the
> manual.

Between:
*****
If you want to temporarily disable a test block, put # in place of the
block type. This creates a comment block which is echoed in the log
file but not executed. For example:

     %!#demo
     %! t = [0:0.01:2*pi]; x = sin (t);
     %! plot (t, x);
     %! # you should now see a sine wave in your figure window"
*****

and:
*****
"Block type summary:"
*****

I would insert:

*****
The following trivial code snippet provides examples for the use of
fail, assert, error and xtest:

function output = mustBeZero(input)
if (input ~= 0)
  error('Non-zero input!')
end
output = input;

%!fail('mustBeZero(1)');
%!assert(mustBeZero(0), 0);
%!error <Non-zero> mustBeZero(1);
%!xtest error('This code generates an error');

Output in response to:

            test mustBeZero verbose

Is:

            >>>>><path to>\mustBeZero.m
              ***** fail('mustBeZero(1)');
              ***** assert(mustBeZero(0), 0);
              ***** error <Non-zero> mustBeZero(1);
              ***** xtest fail(error('This code generates an error'));
            !!!!! known failure
            This generates an error
            PASSES 4 out of 4 tests (1 expected failures)
*****

And then in the section for fail:

Change:
*****
— Function File: fail (code)
— Function File: fail (code, pattern)
— Function File: fail (code, 'warning', pattern)
Return true if code fails with an error message matching pattern,
otherwise produce an error. Note that code is a string and if code
runs successfully, the error produced is:

                    expected error but got none
If the code fails with a different error, the message produced is:
*****

To:
*****
— Function File: fail (code)
— Function File: fail (code, pattern)
— Function File: fail (code, 'warning', pattern)

If called with one argument, return true if code fails.  If code runs
successfully, the error produced is:

                    expected error <.> but got none

NOTE: code must be in the form of a string that may be passed by fail
to the Octave interpreter via the evalin function (i.e. a (quoted)
string constant or a string variable).

If called with two arguments, the behavior is similar to fail(code),
except the return value will only be true if code fails with an error
message containing pattern (case sensitive).  If the code fails with a
different error to that given in pattern, the message produced is:
*****

Thoughts?

On a related note, the documentation for evalin contains the text:
"One example is the fail (‘code’, ‘pattern’) function which evaluates
‘code’ in the caller's context and checks that the error message it
produces matches the given pattern":
http://www.gnu.org/software/octave/doc/interpreter/Evaluation-in-a-Different-Context.html#doc%2devalin

So I'm not totally sure what the convention for showing strings in the
function documentation is (should be?) to make my suggestion above
consistent with the remainder of the documentation.

Thanks for your help,

Iain
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: TDD in Octave, guidance/tutorial?

IainIsm
In reply to this post by Jordi Gutiérrez Hermoso-2
On 5 February 2013 16:35, Iain Cunningham  wrote:
>
> On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave]
> <[hidden email]> wrote:
> > On 5 February 2013 07:01, IainIsm <[hidden email]> wrote:
> >
> >> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave]
> [snip]
[snip]
> Thoughts?

Did you get a chance to consider the proposal at all please Jordi?

Iain
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: TDD in Octave, guidance/tutorial?

Jordi Gutiérrez Hermoso-2
On 26 February 2013 11:15, IainIsm <[hidden email]> wrote:

> On 5 February 2013 16:35, Iain Cunningham  wrote:
>>
>> On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave]
>> <[hidden email]> wrote:
>> > On 5 February 2013 07:01, IainIsm <[hidden email]> wrote:
>> >
>> >> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave]
>> [snip]
> [snip]
>> Thoughts?
>
> Did you get a chance to consider the proposal at all please Jordi?

Sorry for the delay. I have put your suggestions on the stable branch:

    http://hg.savannah.gnu.org/hgweb/octave/rev/7fe4ea72ba4dw

Thanks for the contribution.

- Jordi G. H.
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: TDD in Octave, guidance/tutorial?

Jordi Gutiérrez Hermoso-2
On 4 March 2013 16:53, Jordi Gutiérrez Hermoso <[hidden email]> wrote:

> On 26 February 2013 11:15, IainIsm <[hidden email]> wrote:
>> On 5 February 2013 16:35, Iain Cunningham  wrote:
>>>
>>> On 5 February 2013 13:59, Jordi Gutiérrez Hermoso-2 [via Octave]
>>> <[hidden email]> wrote:
>>> > On 5 February 2013 07:01, IainIsm <[hidden email]> wrote:
>>> >
>>> >> On 4 February 2013 18:08, Jordi Gutiérrez Hermoso-2 [via Octave]
>>> [snip]
>> [snip]
>>> Thoughts?
>>
>> Did you get a chance to consider the proposal at all please Jordi?
>
> Sorry for the delay. I have put your suggestions on the stable branch:
>
>     http://hg.savannah.gnu.org/hgweb/octave/rev/7fe4ea72ba4dw

Sorry again, bad URL:

    http://hg.savannah.gnu.org/hgweb/octave/rev/7fe4ea72ba4d

- Jordi G. H.
_______________________________________________
Help-octave mailing list
[hidden email]
https://mailman.cae.wisc.edu/listinfo/help-octave
Loading...