Unit testing

Testing is how we verify that a program works as we intends it to.

Unit testing is the most elementary form of testing. In this type of testing, the smallest code units are tested individually and in isolation. A code unit is the smallest part of a program that can take input and yield output, this includes functions, methods and other callable objects..

Unit testing ensures that an error encountered in one code unit will not be propagated to tests of other units, this makes locating and correcting bugs easier as we can easily single out the erroneous unit.

We can perform unit testing with two builtin libraries:

Doctest allows us to combine documentation with testing. While it is an effective way of performing unit testing, it is not very suitable for writing long and complex tests. That is why we have unittest.

The unittest framework provides the necessary tools for writing and running unit tests. The framework is readily available in the standard library, we therefore don't have to perform any preliminary installations before using it.

To understand how unit test works, It is important that you be conversant with the following topics:

Unit testing and testing in general is highly reliant on assertion logic. Basically,  assertion involves making a claim and then trying to falsify or prove it.

assert 5 * 2 == 10, 'Not True'

In the above case the assertion passes therefore no exception is raised.

assert 5 * 2 == 20, 'Not True'

In the above case, the assertion fails because 5 * 2 is not equal to the expected value which is 20.

In unittest we use assertions a lot but in a functional way i.e using functions/methods. 

Basic Unit test example

Before we dive deeper on how unit tests work, let us start with a working example.

In the following example, we will define some functions for calculating areas of various shapes. We will then write the unit tests in a class called TestAreas.

#areas.py

import unittest

def circle(r):
   return 3.14 * r * r 

def triangle(b, h):
   return 0.5 * b * h 

def  rectangle(l, w):
   return l * w 

#the tests
class TestAreas(unittest.TestCase):
   def test_circle(self):
      self.assertEqual(circle(r = 7), 153.86)
      self.assertEqual(circle(r = 10), 314.0)

   def test_triangle(self):
      self.assertEqual(triangle(b =8, h = 6), 24)
      self.assertEqual(triangle(b = 3, h = 4), 6.0)

   def test_rectangle(self):
      self.assertEqual(rectangle(l = 4, w = 5), 20)
      self.assertEqual(rectangle(l = 8, w = 10), 80)

if __name__ == '__main__':
   unittest.main()

...
----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

The above example, demonstrates the basics of how unittest works. Basically, the tests are written in a class which inherits from unittest.TestCase class. Tests for individual components are implemented as methods of the class.

The test class inherits some useful assertion methods from TestCase. In the above example, we used the assertEqual() method. The operation of the assertEqual and other inherited assertion methods should be easy to understand as they basically act like the standard assert statement albeit in a functional way. For example, the assertEqual method checks for equality between two values.  If an assertion fails in a given test method, that particular test is marked as a failure.

Consider the following example, where some test fails.

#areas.py

import unittest

def circle(r):
   return 3.14 * r * r 

def triangle(b, h):
   return 0.5 * b * h 

def  rectangle(l, w):
   return l * w 

#the tests
class TestAreas(unittest.TestCase):
   def test_circle(self):
      self.assertEqual(circle(r = 7), 100) #this will fail
      self.assertEqual(circle(r = 10), 314.0)

   def test_triangle(self):
      self.assertEqual(triangle(b =8, h = 6), 24)
      self.assertEqual(triangle(b = 3, h = 4), 6.0)

   def test_rectangle(self):
      self.assertEqual(rectangle(l = 4, w = 5), 200) # this will fail
      self.assertEqual(rectangle(l = 8, w = 10), 80)

if __name__ == '__main__':
   unittest.main()

FF.
======================================================================
FAIL: test_circle (__main__.TestAreas.test_circle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\John\Desktop\test.py", line 17, in test_circle
    self.assertEqual(circle(r = 7), 100) #this will fail
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 153.86 != 100

======================================================================
FAIL: test_rectangle (__main__.TestAreas.test_rectangle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\John\Desktop\test.py", line 25, in test_rectangle
    self.assertEqual(rectangle(l = 4, w = 5), 200) # this will fail
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 20 != 200

----------------------------------------------------------------------
Ran 3 tests in 0.002s

FAILED (failures=2)

How to run the tests

We can use two approches to run the unit tests:

  • Programmatically using unittest.main()
  • Using the unittest commandline tool.

We have used the first approach for running tests in the previous example. In this approach you can execute the tests like a regular python file in your text editor/IDE. Or in the shell/cmd with the following command.

python <yourfile.py>

If you observe the previous examples, you will see that the unittest.main() function is called inside of  if __name__ = __main__:  , this may may be unfamiliar. But it simply makes sure that the tests are only run if the file is executed directly as the main script and not if it is being executed through importation.

In the second approach, the tests are executed in the commandline/shell with unittest command, in this approach we do not need to explicitly call unittest.main()unittest will do this for us in the background. The basic command is as shown below:

python -m unittest <yourfile.py>

Separating tests with main file

In most practical cases, it is usually more preferable to put tests in another file of their own and if the tests are many, you can even divide the tests into multiple test files.

For example, in our previous example, we can create another file called test_areas.py, and put the tests there. Our project structure will look as follows:


├── areas.py
└── test_areas.py

The functions will remain in areas.py, it will now look as follows:

#areas.py

def circle(r):
   return 3.14 * r * r 

def triangle(b, h):
   return 0.5 * b * h 

def  rectangle(l, w):
   return l * w 

We can now import the functions in areas.py in test_areas.py

#test_areas.py

from areas import *

class TestAreas(unittest.TestCase):
   def test_circle(self):
      self.assertEqual(circle(r = 7), 153.86)
      self.assertEqual(circle(r = 10), 314.0)

   def test_triangle(self):
      self.assertEqual(triangle(b =8, h = 6), 24)
      self.assertEqual(triangle(b = 3, h = 4), 6.0)

   def test_rectangle(self):
      self.assertEqual(rectangle(l = 4, w = 5), 20)
      self.assertEqual(rectangle(l = 8, w = 10), 80)

if __name__ == '__main__':
   unittest.main()

...
----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

Separating tests from the main file avoids cluttering the main file, it is also much easier to manage and maintain the tests when they are in their own file.

The inherited assertion methods

Assertion methods are used to compare results with expected values. If the results do not match with the expected values, that particular test is considered a failure.

In the previous example, we used the assertEqual() method. This method simply checks if the returned result is equal to the expected value. There are quite a number of similar methods, some of them are listed in the following table with their builtin equivalent where possible.

assertEqual(a, b) a == b
assertTrue(x) bool(x) == True
assertFalse(x) bool(x) == False
assertIsNone(x) x is None
assertIs(a, b) a is b
assertIn(a, b) a in b
assertIsInstance(a, b) isinstance(a, b)
assertRaises(e)  

There is a counter-method for almost each of the methods in the above list e.g assertNotEqualassertIsNot, assertNotIsinstance, e.t.c

Creating test fixtures

Fixtures are used to perform preparatory tasks at the beginning of the tests and final tasks after a test is done with execution. The preparatory tasks are also known as setup operation while the finalizing tasks are known as teardown operations.

The setup operations may involve initializing the necessary data that is to be used in the tests,  this may include creating objects, opening files or database e.tc. Teardown operations are used for clean up like destroying created objects, closing opened files and other connections, e.t.c.

Unittest provide several fixtures methods, the two most basic are setUp() and tearDown(). The two methods are called automatically before and after running each test.

For example, consider if instead of defining the functions for calculating areas each on its own, we instead implement them as  methods of a class called Area as shown below:

#areas.py

class Area:
   def circle(self, r):
      return 3.14 * r * r 

   def triangle(self, b, h):
      return 0.5 * b * h 

   def  rectangle(self, l, w):
      return l * w 

When testing the class, we can create the test instance in the setUp() method, then delete the instance in the tearDown() method.

#the tests

import unittest
from foo import Area

class TestAreas(unittest.TestCase):
   def setUp(self):
      self.area = Area()
   def tearDown(self):
      del self.area 

   def test_circle(self):
      self.assertEqual(self.area.circle(r = 7), 153.86)
      self.assertEqual(self.area.circle(r = 10), 314.0)

   def test_triangle(self):
      self.assertEqual(self.area.triangle(b =8, h = 6), 24)
      self.assertEqual(self.area.triangle(b = 3, h = 4), 6.0)

   def test_rectangle(self):
      self.assertEqual(self.area.rectangle(l = 4, w = 5), 20)
      self.assertEqual(self.area.rectangle(l = 8, w = 10), 80)

if __name__ == '__main__':
   unittest.main()

...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

Note that ,  the above approach is more convenient and clean than manually creating an Area object in each test method.

Apart from setUp() and tearDown() that we have seen above, unittest offers more fixtures. The following table highlights the various fixture and when they are called. 

.setUp() An instance method. Called before each test is executed.
.tearDown() An instance method. Called after each test has been executed.
.setUpClass() A class method. Called before each tests is executed.
.tearDownClass() A class method. Called after each test has been executed.

The setUpClass and tearDownClass are class methods, we therefore need to use the @classmethod decorator when implementing them.

class TestClass(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
       ...
    @classmethod
    def tearDownClass(cls):
       ...

A more practical Unit test example

In these part we will implement  a stack data structure then write its tests in a separate file.

A stack is a data structure that follows the LIFO approach. This means that the latest element is retrieved first.

Operations on a stack happens in one end which is known as the top of the stack.

A stack typically supports the following operations:

  • push(x) - add element x at the top of the stack.
  • pop() - Remove and return the element at the top of the stack.
  • top() - Return but don't remove the element at the top.
  • isEmpty() - Check whether the stack is empty.
  • len() - Return the number of elements in the stack.

We will use a list as the underlying data structure for storing the elements. 

#stack.py

class Stack:
   def __init__(self):
      self._data = [] #list for storing the elements
      self._size = 0  #initial size

   def __len__(self):
      return self._size

   def isEmpty(self):
      return self._size == 0

   def top(self):
      if self.isEmpty():
         raise Empty('Stack is Empty.')

      return self._data[-1]

   def push(self, x):
      self._data.append(x)
      self._size += 1

   def pop(self):
      if self.isEmpty():
         raise Empty('Stack is Empty.')

      self._size -= 1
      return self._data.pop()

In this example, we started by writing the class itself, however, it is also common to write the tests first and actual implementations later.  Let us now write the tests and run them.

#test_stack.py

import unittest

from foo import Stack, Empty

class TestStack(unittest.TestCase):
   def setUp(self):
      self.stack = Stack()

   def tearDown(self):
      del self.stack

   def test_isEmpty(self):
      self.assertTrue(self.stack.isEmpty()) #the stack is initially empty

      self.stack.push(100) #push an item.
      self.stack.push(200)
      self.assertFalse(self.stack.isEmpty()) #the stak is not empty after pushing

      self.stack.pop()
      self.stack.pop()
      self.assertTrue(self.stack.isEmpty()) #the stack is empty again
   
   def test_length(self):
      self.assertEqual(len(self.stack), 0) #initially 0

      #add items
      self.stack.push(100)
      self.stack.push(200)
      self.stack.push(300)
      self.assertEqual(len(self.stack), 3) #should have a length of 3

      self.stack.pop()
      self.stack.pop()
      self.stack.pop()
      self.assertEqual(len(self.stack), 0) #should be 0 again


   def test_push(self):
      self.stack.push(10)
      self.assertEqual(self.stack.top(), 10)

      self.stack.push(20)
      self.assertEqual(self.stack.top(), 20)

      self.stack.push(30)
      self.assertEqual(self.stack.top(), 30)

   def test_pop(self):
      with self.assertRaises(Empty):
         self.stack.pop()  #the stack is empty


      self.stack.push(100)
      self.stack.push(200)
      self.stack.push(300)

      self.assertEqual(self.stack.top(), self.stack.pop())
      self.assertEqual(self.stack.top(), self.stack.pop())
      self.assertEqual(self.stack.top(), self.stack.pop())


      with self.assertRaises(Empty):
         self.stack.pop() #empty again

if __name__ == '__main__':
   unittest.main()

....
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

In the above example, we made sure that all the tests pass. You should try tinkering with the Stack class to see the tests fail. For example, you can remove some statements in the Stack methods to see the effect they will have on the tests.