pinocchio -- extensions for the 'nose' unit testing framework

Contents

Introduction

pinocchio is a set of extensions to the nose unit testing framework for Python.

The current version of pinocchio is 0.1, available at

http://darcs.idyll.org/~t/projects/pinocchio-0.1.tar.gz

You can always get the latest pinocchio distribution at

http://darcs.idyll.org/~t/projects/pinocchio-latest.tar.gz

You must install pinocchio with easy_install, e.g.

easy_install http://darcs.idyll.org/~t/projects/pinocchio-latest.tar.gz

pinocchio only works with nose 0.9a1 and above.

Extensions

stopwatch -- selecting tests based on execution time

Sometimes your unit tests just seem to take forever. Well, now you can get rid of the slow ones automatically!

The pinocchio.stopwatch extension module lets you time the unit tests being run, and -- once times have been recorded -- then lets you select only those that run faster than a given amount of time. As a bonus, the test names and run times are stored in a simple format -- a pickled dictionary -- so you can target specific tests for speedup, too.

Use cases

There's really only one use case here: your tests take too long to run, so you and your developers aren't running them very frequently. The stopwatch module lets you pick out the fast ones to be run via the command line; you can always run the slow ones in your continuous build system, right?

Options

--with-stopwatch enables test timing.

--stopwatch-file changes the filename used to save the pickled test times from '.nose-stopwatch-times' to the specified file.

--faster-than sets an upper time limit (in seconds) for which tests should be run.

Examples

See examples/test_stopwatch.py for some examples; use

nosetests -w examples/ --with-stopwatch --faster-than 1 test_stopwatch.py
nosetests -w examples/ --with-stopwatch --faster-than 1 test_stopwatch.py

to run a subset of the tests. Note that you need to run it twice -- once to record the times (i.e. all tests will be run, independent of the --faster-than parameter) and again to select only the "fast" tests.

decorator -- adding attributes to tests

The attrib extension module for nose is a great way to select subsets of tests based on attributes you've given the test functions, classes, or methods. But what if you don't want to modify the source code to add the attributes?

The pinocchio.decorator extension module lets you provide the names of functions, classes, and methods to which to add tags. For example,

TestModule.test_function: a
TestModule.TestClass: b
TestModule.TestClass.test_method: c

would set attributes on a function, a class, and a method. Then

nosetests -a a

would run only the function,

nosetests -a b

would run all methods on the class, and

nosetests -a c

would run only the method.

Use cases

There are a couple of scenarios where this can come in handy:

  • You're working on a bunch of tests that are failing, and you only want to execute the failing tests. If modifying the failing test code itself would be more work than simply listing them in a file, then use the decorator extension.
  • You have a 'private' set of unit tests that bear on the code you're working on. You want to iterate quickly, just running those unit tests. Again, if modifying the test code itself is more work than simply listing the relevant tests in a file, use the decorator extension.
  • You have a few unit tests that are failing, and you don't want to execute them (that is, the reverse of the first scenario).

Options

--decorator-file specifies the file containing the tags to use.

Examples

See examples/test_decorator.py for some examples; use examples/test_decorator.attrib as the decorator file. For example, try the following commands:

nosetests -w examples/ --decorator-file examples/test_decorator.attribs examples/test_decorator.py -a one
nosetests -w examples/ --decorator-file examples/test_decorator.attribs examples/test_decorator.py -a two
nosetests -w examples/ --decorator-file examples/test_decorator.attribs examples/test_decorator.py -a three

figleafsections -- find out what tests are executing which parts of your code

(You'll need to install figleaf to use this plugin; it will install the figleaf package and the annotate-sections script.)

This plugin lets you record code coverage per unit test, and then annotate your Python source code with which unit tests are running which lines of code. It's a useful way to figure out which nose tests are exercising what parts of your program.

See http://ivory.idyll.org/blog/feb-07/figleaf-goodness.html for some detailed examples.

To try it out, do this:

nosetests -w examples/ --with-figleafsections examples/test_sections.py
annotate-sections examples/test_sections.py

The output will be placed in the file examples/test_sections.py.

outputsave -- save your stdout into files

This plugin records the stdout from each test into a separate file, with a prefix indicating whether or not the test succeeded.

Use cases

The main use case is when you have MANY failing tests and you want to take a look at the output without having to page through the nose error output linearly.

Options

--with-outputsave enables the plugin. Output from successful tests will be placed in success-<testname>, output from failed tests will be placed in fail-<testname>, and output from errors will be placed in error-<testname>.

--omit-success does not save output from successful tests, i.e. only 'fail-' and 'error-' output files will be created.

--save-directory places all saved output into the given directory, creating it if it does not exist.

Examples

Try:

nosetests -w examples/ --with-outputsave --save-directory=output examples/test_outputsave.py

Then look at the 'output' directory.

spec -- generate test description from test class/method names

spec lets you generate a "specification" similar to testdox . The ppec plugin can generate simple documentation directly from class and method names of test cases. For example, a test case like:

class TestFoobar:
    def test_is_a_singleton(self):
        pass
    def test_can_be_automatically_documented(self):
        pass

during the test run will generate the following specification:

Foobar
- is a singleton
- can be automatically documented

Test functions put directly into a module will have a context based on the name of the containing module. For example, if you define functions test_are_marked_as_deprecated() and test_doesnt_work_with_sets() in a module test_containers.py, you'll get the following specs:

Containers
- are marked as deprecated
- doesn't work with sets

Use cases

If you follow a good naming convention for your tests you'll get free up-to-date specification of your application - it will be as accurate as your tests are.

Options

--with-spec enables the plugin, and automatically sets the verbose level for nose to "detailed output". During the test run all your test descriptions will be shown as a special kind of specification - your test classes set up a context and test methods set up a single specification.

--spec-color enables colored output. Successful tests will be marked as green, while failed/error cases as red. Skipped and deprecated test cases will be shown in yellow. You need an ANSI terminal to use this.

--spec--doctests enables experimental support for doctests.

Examples

Try:

nosetests --with-spec --spec-color examples/test_spec.py

(You should see an error, yes.)

Look at examples/test_spec.py source code and tests inside tests/spec_test_cases/ directory to see how test cases are mapped into specifications.

License

pinocchio is available under the MIT license.

Author Information

The author of the stopwatch, decorator, figleafsections, and outputsave extensions is Titus Brown. You can contact him at titus@idyll.org, or check out his main site at http://ivory.idyll.org/.

The author of the spec plugin is Michal Kwiatkowski. His homepage is at http://joker.linuxstuff.pl/ and his mail address is constant.beta@gmail.com.