functions missing tests: goal of tests?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

functions missing tests: goal of tests?

msander

It has been asserted in a few threads recently that writing tests is a
good entry point to octave development.

I have consulted some online resources to determine how to go about
writing said tests.  Specifically http://wiki.octave.org/Tests has some
good information.  I was thinking of tackling scripts wiithin
scripts/general & scripts/miscellaneous as a starting point.

Software testing is a rather broad field.  "write tests for untested
functions" is a rather vague notion.  I am trying to get my head around
"what are the requirements?", "what needs to be tested"? etc.  I am
putting out a call for some guidance in this regard.

As i understand it, the tests within m files (and probably cc) are unit
tests... and probably within the "grey box" testing category.    ie.
testing the code for various inputs to produce desired outputs, while
having some knowledge of the inner workings of said code.

Given the above I believe the purpose of the tests with m files is to
test correct behavior of these functions across a representative set of
data types / data values.  This includes returning an error for invalid
inputs.   Is this consistent with how other maintainers view m file
tests?   Should the tests include other criterion?   An example could be
100% code coverage.  Personally, I would include 100 % code coverage as
a future goal.

Are there other requirements at a function level that I may be
missing?   Ideally tests should capture those as well.   A link to said
documentation would be appreciated.
Reply | Threaded
Open this post in threaded view
|

Re: functions missing tests: goal of tests?

Michael Godfrey
On 01/12/2014 10:03 PM, mike sander wrote:
> Are there other requirements at a function level that I may be
> missing?   Ideally tests should capture those as well.   A link to
> said documentation would be appreciated.
I think you get the idea quite well.  It would be useful for you to look
at a number of the source files which have tests. All of the tests are
contained within each file, usually at the bottom.  You will find a range
of testing quality and coverage.  This points out that the problem is
not just that many files have no tests, but many have quite incomplete
tests. You should feel free to work on those, too.

Thanks for offering to help.

Michael

Reply | Threaded
Open this post in threaded view
|

Re: functions missing tests: goal of tests?

fgnievinski
In reply to this post by msander
I think all requirements follow from code coverage, meaning
Once you've reached 100% there's nothing more to check.

I think it should always be assumed that the functions called
are reliable -- otherwise it's them that need better testing.

Sometimes you might find that splitting up a given function
into sub-functions reduces the combinatorial coverage, e.g.:

function out = a (in)
   out = b (c (in));
end

Then if you test b and c separately, you don't need to test
all the combinations of input for b and c.

-F.