Unittest template

Recently, I have to test the same function with only few differences. For example, I have to test something like that:

def support_http_method(method):
    if method in ('POST', 'GET'):
        return True
    else:
        return False

This kind of function are easy to test, you give them an input and you check the output. So let's test this function:

class SupportHttpMethodTestCase(unittest.TestCase):

    def test_post(self):
        self.assertTrue(support_http_method('POST'))

    def test_get(self):
        self.assertTrue(support_http_method('GET'))

    def test_put(self):
        self.assertFalse(support_http_method('PUT'))

    def test_delete(self):
        self.assertFalse(support_http_method('DELETE'))

Problem

This test file is quite redundant, isn't it? How can you remove this code duplication because code duplication is the root of all evil, don't forget it!

First solution

The most simple idea is to use a loop in the test method, like that:

class SupportHttpMethodTestCase(unittest.TestCase):

    def test_support(self):
        expected_values = {
            'POST': True,
            'GET': True,
            'PUT': False,
            'DELETE': False
        }
        for method, expected in expected_values.items():
            self.assertEqual(support_http_method(method), expected)

This solution is quite good because it removes code duplication, but you should really never do that.

The problem

The reason is simple: because the unit test library will consider your method as one test. What it mean exactly? Let's launch the previous code:

$ python test_first.py
.
----------------------------------------------------------------------
Ran 1 test in 0.006s

OK

Now let's introduce a typo in the function:

def support_http_method(method):
    if method in ('POST', 'GETT'):
        return True
    else:
        return False

And rerun the test with the broken code:

$ python test_first_broken.py
F
======================================================================
FAIL: test_support (__main__.SupportHttpMethodTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_first_broken.py", line 19, in test_support
    self.assertEqual(support_http_method(method), expected)
AssertionError: False != True

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

You see, you don't know which test has failed (it's normal as you have only one test), you don't know with what parameters the function was called and you don't even know if the method works with other arguments.

Worse, as you have only one test, setUp and tearDown will be called only one time.

Conviced? It's a very bad solution, but you want to do something like that, let's see a better solution.

A better solution

A better solution is to extract the loop and place it outside the test. Don't worry, you will not do it yourself, you will use nosetest test generator. You should check the documentation to understand how it works, it's quite similar to PHPUnit data provider.

Let's test our function with this test generator:

def test_support():
    expected_values = {
        'POST': True,
        'GET': True,
        'PUT': False,
        'DELETE': False
    }
    for method, expected in expected_values.items():
        yield check_support, method, expected

def check_support(method, expected):
    assert support_http_method(method) == expected

(Nosetest does not support test generator with unittest.TestCase classes, so we must use test functions instead.)

Let's run the generator code:

$ nosetests
....
----------------------------------------------------------------------
Ran 4 tests in 0.004s

OK

And let's broke the function:

$ nosetests -v
test_first.test_support('PUT', False) ... ok
test_first.test_support('POST', True) ... ok
test_first.test_support('DELETE', False) ... ok
test_first.test_support('GET', True) ... FAIL

======================================================================
FAIL: test_first.test_support('GET', True)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/private/tmp/test_second_broken.py", line 18, in check_support
    assert support_http_method(method) == expected
AssertionError

----------------------------------------------------------------------
Ran 4 tests in 0.004s

FAILED (failures=1)

What is the difference? You have four different tests, you know that the test has failed with 'GET' method and that other methods work fine. It's a great advance over the previous solution.

But, the non-support for class-based tests and the syntax verbosity pushed me to develop another solution.

My solution

What I miss in the previous solution?

  • Support for class-based tests.
  • Less verbose declaration.

It's why I develop unittest templates, let's see how you can test the function:

from poc import TemplateTestCase, Call, template

class SupportHttpMethodTestCase(unittest.TestCase):

    __metaclass__ = TemplateTestCase

    support_args = {
        'post': Call('POST', True),
        'get': Call('GET', True),
        'put': Call('PUT', False),
        'delete': Call('DELETE', False)
    }

    @template(support_args)
    def _test_support(self, method, expected):
        self.assertEquals(support_http_method(method), expected)

Analyze all steps necessary to use unittest templates:

  • Set the metaclass, it's necessary as the metaclass will create tests.
  • Defines your test's arguments as a class attribute.
  • Decorate your test with template decorator and pass it the test's arguments.
  • Run it as you want (directly, using nose, etc...).

Let's try to launch our test template:

$ python test_first.py -v
test_support_delete (__main__.SupportHttpMethodTestCase) ... ok
test_support_get (__main__.SupportHttpMethodTestCase) ... ok
test_support_post (__main__.SupportHttpMethodTestCase) ... ok
test_support_put (__main__.SupportHttpMethodTestCase) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

And with nose:

$ nosetests -v
test_support_delete (test_first.SupportHttpMethodTestCase) ... ok
test_support_get (test_first.SupportHttpMethodTestCase) ... ok
test_support_post (test_first.SupportHttpMethodTestCase) ... ok
test_support_put (test_first.SupportHttpMethodTestCase) ... ok

----------------------------------------------------------------------
Ran 4 tests in 0.007s

OK

Let's broke our function (again) and see how the template react:

$ python test_first.py
.F..
======================================================================
FAIL: test_support_get (__main__.SupportHttpMethodTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/private/tmp/poc.py", line 15, in wrapped
    return func(self, *(args + parameters), **kw)
  File "test_third_broken.py", line 23, in _test_support
    self.assertEquals(support_http_method(method), expected)
AssertionError: False != True

----------------------------------------------------------------------
Ran 4 tests in 0.001s

FAILED (failures=1)

It's the same behaviour as with nosetest test generator.

One extra versus nosetest test generator, you can use test's arguments with different test method.

The code is still in beta phase, there is no complete doc for it, but we use it every day at dailymotion where I do my internship.

Thank you for reading this article, one more thing before leaving, which syntax did you prefer between adding test's arguments in it's name (nosetest test generator) or choose a suffix (unittest templates)?

If you really like the test templates ideas and implementation you can leave a comment to show me this piece of code is really helpful and I will try to incorporate this little piece of code (~40 loc) in a larger project (in unittest2 or nosetests ideally).

Comments !