Introduction to testing in Python

Testing is how we prove that our code is reliable and does exactly what we want. It is a very important aspect of software development as it gives us confidence that a piece of code and the entire program is reliable and predictable under all possible circumstances.

There are a lot of ways that we can test our code. In fact, you have most likely engaged in some form of testing even if unknowingly. Remember that time you printed out the results of a function to see if the function was working as intended? well that is known as manual testing. While manual testing may work for small pieces of code, it is highly limited when it comes to real-life applications. This is why we need a well structured form of testing which is achieved through automated testing.

In automated testing, we do not have to manually inspect the operations of our code to tell whether it is working as expected, instead,  we write chunks of test code to do this for us. Fortunately, we do not have to do this from scratch,  there is a set of standard library modules as well as third party libraries and frameworks for writing automated tests.

In this article you will learn how automated testing works and why it is an indispensable skill any programmer should have.

Types of testing

Automated testing can be sub-divided into three categories:

Unit testing is the most fundamental form of testing. In unit testing, we test the smallest parts of a program in isolation. This typically means that testing is done to individual components such as  functions and methods.  It ensures that errors from one unit is not propagated to other parts and we can therefore easily single out an erroneous unit.

Integration test  tests that related components works correctly as a group. If individual components work as expected when tested through unit testing, it does not necessarily confirm that they will work correctly when joined together. This is why integration testing is necessary.

Lastly system testing tests that the entire program/system works correctly. In itself, system testing is heavily reliant on unit testing and integration testing this is because for the larger system to function as expected, smaller parts have to work first. You do not expect your program to work correctly if there is an error in a particular function, method or class.

Basic testing example

One of the most basic testing tool in Python is the assert statement. The statement simply checks whether a given condition is satisfied, if satisfied, it does nothing, otherwise it raises an AsserionError exception. The assert statement has the following syntax:

assert <condition>, <message>

The condition is evaluated, if it passes, nothing is done. The message is optional and will be included in the AssertionError exception if the condition fails.

In the following example, we use the assert statement to create simple tests.

def circle_area(r):
   return 3.14 * r * r  

def test_circle_area():
   assert circle_area(7) ==  153.86, 'circle_area(7) should be  153.86'
   assert circle_area(9) == 254.34, 'circle_area(9) should be 254.34'
   assert circle_area(11) == 379.94, 'circle_area(11) should be 379.94'

   print("All tests passed")
   
test_circle_area()

In the above example, we defined the function circle_area() which returns the area of the circle whose radius is given as argument. We created a function to tests the function with assert statements. In the above case all assertions passed, let us look at a case where an assertion fails leading to the AssertionError exception. 

def circle_area(r):
   return 3.14 * r * r  

def test_circle_area():
   assert circle_area(7) == 150, 'circle_area(7) should be  153.86' #fails
   
test_circle_area()

In the above case the assertion fails, because circle_area(7) is not 150.

Using the assert statement for testing is okay for performing simple checks. There are more fully-fledged and convenient testing libraries and frameworks for creating and running tests. 

Documentation testing with doctest

We can write tests as a part of documentation. They tests are then executed using the doctest module in the standard library.

The test statements are entered in the docstring as if they were copied from an interactive session in a shell.

Consider the following example:

#foo.py

def circle_area(r):
   '''
   >>> circle_area(7)
   153.86
   >>> circle_area(9)
   254.34
   >>> circle_area(11)
   100
   '''

   return 3.14 * r * r  

In the above example, we have a module called foo.py, we then defined the function circle_area(). Note how the docstring of this function appears. Those are actually valid tests. To run the tests we call doctest from the shell as shown below:

python -m doctest foo.py

If you run the above command in the shell you will get the output similar to one shown below:

As you can see above, doctest found our tests and executed them. The message indicates that one test failed.

doctest allows us to blend tests with regular texts, only lines that look as if they are from an interactive session are treated as tests.

​
#foo.py

def circle_area(r):
   '''
   This is an example
   To show that we can use regular text
   with doctests in a docstring
   >>> circle_area(7)
   153.86
   >>> circle_area(9)
   254.34
   >>> circle_area(11)
   100
   '''

   return 3.14 * r * r  

To run the tests, call the doctest module in verbose mode with the following command.

python -m doctest foo.py -v

The output will be as follows:

As shown above, all the tests passed. Note that the regular text was entirely ignored.

unit testing with unittest

There are many testing frameworks in Python. The three most commonly used are as outlined below:

unittest exists in the standard library and we therefore don't need to perform extra installation to use it. The other two are third party and you have to install them first. 

Most of third party libraries follows the principles of unittest, knowledge on unittest is therefore applicable on other frameworks like nose and pytest.

unittest follows the object oriented paradigm. The tests are written as methods of a class that inherits from a unittest class. Consider the following example:

import unittest

def circle(r):
   return 3.14 * r * r  

def triangle(b, h):
   return 0.5 * b * h

def rectangle(w, l):
   return w * l 

class TestAreas(unittest.TestCase):

   def test_circle(self):
      self.assertEqual(circle(7), 153.86, "circle(7) Should be 153.86")
      self.assertEqual(circle(9), 254.34, "circle(9) Should be 254.34")
      
   def test_triangle(self):
      self.assertEqual(triangle(6, 8), 24, "triangle(6, 8) should be 24")
      self.assertEqual(triangle(3, 5), 7.5, "triangle(3, 5) should be 7.5")

   def test_rectagle(self):
      self.assertEqual(rectangle(4, 5), 20, 'rectangle(4, 5) should be 20') 
      self.assertEqual(rectangle(6, 10), 60, 'rectangle(6, 10) should be 60') 

if __name__ == '__main__':
   unittest.main()   

In the above example, we defined functions for calculating areas of various shapes. We then created the TestAreas class which inherits from unittest.TestCase. The tests for individual functions are implemented as methods of this class.

We executed the tests programmatically by calling unittest.main(). However, you can also call unittest from the shell/cmd, with the following syntax:.

python -m unittest foo.py

unittest will automatically identify and execute methods that begin with "test" e.g test_something.

The unittest.TestCase class have a lot of methods for testing purposes. In the above example, we only used the assertEqual method. However, there are many TestCase methods some of which are listed below.

And many others.....

Benefits of testing

Testing your code is the only way that you can prove and have confidence that your program works correctly.

Untested code is said to be broken by design.

Assuming that a piece of code works correctly without tests is relying on chance. You have to ensure that your code is reliable under all circumstances. This may seem trivial for small personal programs, but in real world it can easily cause costly damages.

Some benefits of testing are outlined below: