Writing Doctests image

Doctest  is the easiest way of performing automated testing in our code. It makes it possible to document and test code at the same time.

Before we look on how to write doctests, it is important to first understand docstrings in Python. Docstrings or documentation strings are simply triple quoted strings that appear as the first statements of a code unit such as a function, class, modules, e.t.c. A docstring typically describe how the code works, how to use it, and any other helpful information that will make the users understand it better.

We can retrieve the docstring of a particular object using the __doc__ attribute. The builtin help() function also includes the docstring in the displayed help text.

documenting a function

def circle_area(r):
   '''Calculates and returns the area of a circle
      whose radius is given.'''

   area = 3.14 * r * r 
   return area

print(circle_area.__doc__, end = '\n\n')
help(circle_area)

In the above example, we defined the circle_area() function which calculates the area of a circle. The function's docstring is the triple quoted text in its body. We printed out the docstring using the __doc__ attribute as well as with the builtin help() function.

Basic example of Doctest

Doctests are written as a part of the docstring. The basic idea is to verify that the piece of code works correctly by writing statements and the expected outputs in the format of the interactive shell session.

The tests are then executed using the builtin module similarly called doctest.

Consider the following basic example:

#foo.py

def circle_area(r):
   '''
   >>> circle_area(r = 7)
   153.86
   >>> circle_area(r = 9)
   254.34
   >>> circle_area(r = 15)
   706.5
   '''

   area = 3.14 * r * r 
   return area

In the above example, we have a file called foo.py where we defined the function circle_area(). Look carefully at the function's docstring, it looks as  if its contents have been copied directly from an interactive session. Those are actually valid tests. Each statement beginning with >>>  is a function call and the subsequent line is the expected output.

When we run the tests, Python executes any statement that begins with >>> and compares its return value with the expected value.

To run the tests:

  • Open the shell/cmd and change directory to where the file foo.py is.
  • Run the following command.
  • python -m doctest foo.py -v
  • The doctest framework will automatically retrieve the tests and execute them.

If you run the above command, you should see an output similar to one shown below:

Trying:
    circle_area(r = 7)
Expecting:
    153.86
ok
Trying:
    circle_area(r = 9)
Expecting:
    254.34
ok
Trying:
    circle_area(r = 15)
Expecting:
    706.5
ok
1 items had no tests:
    foo
1 items passed all tests:
   3 tests in foo.circle_area
3 tests in 2 items.
3 passed and 0 failed.
Test passed.

The above output indicates that all the tests passed. Consider the following example where some tests fail.

#foo.py

def circle_area(r):
   '''
   >>> circle_area(r = 7)
   150
   >>> circle_area(r = 9)
   254.34
   >>> circle_area(r = 15)
   706.5
   '''

   area = 3.14 * r * r 
   return area

Running the shell command now will lead to the following output.

Trying:
    circle_area(r = 7)
Expecting:
    150
**********************************************************************
File "C:\Users\John\Desktop\foo.py", line 3, in foo.circle_area
Failed example:
    circle_area(r = 7)
Expected:
    150
Got:
    153.86
Trying:
    circle_area(r = 9)
Expecting:
    254.34
ok
Trying:
    circle_area(r = 15)
Expecting:
    706.5
ok
1 items had no tests:
    foo
**********************************************************************
1 items had failures:
   1 of   3 in foo.circle_area
3 tests in 2 items.
2 passed and 1 failed.
***Test Failed*** 1 failures.

 As shown above one test failed because circle_area(r = 7) is not equal to the expected value i.e 150.

The syntax of doctests

As earlier mentioned, doctest identifies tests by looking for parts of the docstring that look as if they were copied  directly from an interactive session.

You have most likely used Python's interactive mode in the shell/cmd. Doctests syntax is just similar to that of text from the interactive session. In fact, you can copy lines directly from the interactive shell and paste them as tests in the docstring and they will work just fine.

The structure of the doctests is as described below:

  • Lines starting with >>> are sent to the interpreter for execution.
  • A line starting with ... is sent as a continuation of the code from the previous line.
  • Bare lines that  do not start with any prefix represents the expected outputs from the previous statement(s).
  • If no output is given, doctest assumes that a statement should not have any visible output in the console.

doctest will ignore any piece of text that is not a part of a tests. This means that we can place regular text in the docstring alongside the tests. Consider the following example:

#foo.py

def circle_area(r):
   '''
   Calculates and returns the area of a circle
      whose radius is given.

   >>> circle_area(r = 7)
   153.86
   >>> circle_area(r = 9)
   254.34
   >>> circle_area(r = 15)
   706.5

   this is the end of documentation
   '''

   area = 3.14 * r * r 
   return area

When you run the python -m doctest foo.py -v command with the above script, the tests will be executed and the regular text ignored. The output will look as follows:

Trying:
    circle_area(r = 7)
Expecting:
    153.86
ok
Trying:
    circle_area(r = 9)
Expecting:
    254.34
ok
Trying:
    circle_area(r = 15)
Expecting:
    706.5
ok
1 items had no tests:
    foo
1 items passed all tests:
   3 tests in foo.circle_area
3 tests in 2 items.
3 passed and 0 failed.
Test passed.

Doctest with statements expecting exceptions

In some circumstances we may expect a piece of code to raise an exception. And we may want to write tests to verify that the exception is raised in such cases.

def circle_area(r):
   if r < 0 :
      raise ValueError(f'A circle with negative radius is undefined.')

   area = 3.14 * r * r 
   return area

print(circle_area(-5))

Traceback (most recent call last):
  File "C:\Users\John\Desktop\foo.py", line 10, in <module>
    print(circle_area(-5))
          ^^^^^^^^^^^^^^^
  File "C:\Users\John\Desktop\foo.py", line 5, in circle_area
    raise ValueError(f'A circle with a negative radius is undefined.')
ValueError: A circle with a negative radius is undefined.

When we want to test that an exception is raised,  it would be inconvenient or even impossible to write the entire expected error message including the tracebacks without copy pasting. Doctest makes it easy to test for exceptions by only requiring us to provide only two things:

  1. The first line i.e Traceback (most recent call last):- This tells doctest that we expect an exception to be raised.
  2. The part after the traceback which indicates which exception was raised. 
#foo.py

def circle_area(r):
   '''
   Calculates and returns the area of a circle
      whose radius is given.

   >>> circle_area(r = 7)
   153.86
   >>> circle_area(r = - 5)
   Traceback (most recent call last):
   ValueError: A circle with a negative radius is undefined.

   End of documentation
   '''

   if r < 0 :
      raise ValueError(f'A circle with a negative radius is undefined.')

   area = 3.14 * r * r 
   return area

After running the doctest command, the output will be as shown below:

Trying:
    circle_area(r = 7)
Expecting:
    153.86
ok
Trying:
    circle_area(r = - 5)
Expecting:
    Traceback (most recent call last):
    ValueError: A circle with a negative radius is undefined.
ok
1 items had no tests:
    foo
1 items passed all tests:
   2 tests in foo.circle_area
2 tests in 2 items.
2 passed and 0 failed.
Test passed.

Testing classes and methods

When defining a class, we can write its tests and those of its method at the top of the class.

#foo.py

class Area:
   '''
   a class for calculating areas for common shapes.

   >>> a = Area()

   #test circle
   >>> a.circle(r = 7)
   153.86
   >>> a.circle(r = 15)
   706.5

   #test triangle
   >>> a.triangle(b = 6, h = 8 )
   24.0
   >>> a.triangle(b = 3, h = 5)
   7.5

   #test rectangle
   >>> a.rectangle(l = 4, w = 5)
   20
   >>> a.rectangle(l = 6, w = 7)
   42
   '''

   def circle(self, r):
      '''
      calculates the area of a circle whose radius is r
      '''

      area = 3.14 * r * r
      return area

   def triangle(self, b, h):
      '''calculates the area of a triangle'''

      area = 0.5 * b * h
      return area

   def rectangle(self, l, w):
      '''calculates the area of a rectangle of the specified length and width'''

      area = l * w 
      return area

If you run the doctest command the output will be as shown below:

Trying:
    a = Area()
Expecting nothing
ok
Trying:
    a.circle(r = 7)
Expecting:
    153.86
ok
Trying:
    a.circle(r = 15)
Expecting:
    706.5
ok
Trying:
    a.triangle(b = 6, h = 8 )
Expecting:
    24.0
ok
Trying:
    a.triangle(b = 3, h = 5)
Expecting:
    7.5
ok
Trying:
    a.rectangle(l = 4, w = 5)
Expecting:
    20
ok
Trying:
    a.rectangle(l = 6, w = 7)
Expecting:
    42
ok
4 items had no tests:
    foo
    foo.Area.circle
    foo.Area.rectangle
    foo.Area.triangle
1 items passed all tests:
   7 tests in foo.Area
7 tests in 5 items.
7 passed and 0 failed.
Test passed.

Doctests in external file

If the tests are many and we do not want to clutter the documentation, we can place the tests or some of them in an external file. We can then execute the tests from the main file.

The doctest module defines the testfile() function which tests an external file from the calling script's scope. Unlike in the other examples where we were running the tests from  the command line, with testsfile() the tests are executed programmatically.

We will create another file called tests.txt where we will place the tests. The working directory should now have a structure as one shown below:

dir
├── foo.py
└── tests.txt

Let us begin by writing the tests. 

#tests.txt

>>> a = Area()

#test circle
>>> a.circle(r = 7)
153.86
>>> a.circle(r = 15)
706.5

#test triangle
>>> a.triangle(b = 6, h = 8 )
24.0
>>> a.triangle(b = 3, h = 5)
7.5

#test rectangle
>>> a.rectangle(l = 4, w = 5)
20
>>> a.rectangle(l = 6, w = 7)
42

We have written the tests for the Area class in the tests.txt file. Let us now define the class in the foo.py file and execute the tests.

#foo.py

class Area:
   '''A class for calculating areas of common shapes.'''

   def circle(self, r):
      '''calculates the area of a circle whose radius is r'''

      area = 3.14 * r * r
      return area

   def triangle(self, b, h):
      '''calculates the area of a triangle'''

      area = 0.5 * b * h
      return area

   def rectangle(self, l, w):
      '''calculates the area of a rectangle of the specified length and width'''

      area = l * w 
      return area

if __name__ == '__main__':
   doctest.testfile('tests.txt', globs = globals(), verbose = True)

Trying:
    a = Area()
Expecting nothing
ok
Trying:
    a.circle(r = 7)
Expecting:
    153.86
ok
Trying:
    a.circle(r = 15)
Expecting:
    706.5
ok
Trying:
    a.triangle(b = 6, h = 8 )
Expecting:
    24.0
ok
Trying:
    a.triangle(b = 3, h = 5)
Expecting:
    7.5
ok
Trying:
    a.rectangle(l = 4, w = 5)
Expecting:
    20
ok
Trying:
    a.rectangle(l = 6, w = 7)
Expecting:
    42
ok
1 items passed all tests:
   7 tests in test.txt
7 tests in 1 items.
7 passed and 0 failed.
Test passed.

In the above example, we used the testfile() function to execute the tests defined in tests.txt file from foo.py. You can call the builtin help() function with the testfile function to see more on its arguments and usage.

import doctest

help(doctest.testfile)

Conclusion:

  • Doctest allows us to document and test a piece of code simultaneously.
  • The tests are written in the docstring of the target objects.
  • The tests should look just as how they would appear in an interactive session.
  • To execute the tests we call the doctest module from shell/cmd with the following command.  
  • python -m doctest <name of the file> <args>
  • After running the  above command, doctest automatically looks for the doctests and executes them.
  • We can also put the tests in an external file then use the doctest.testfile() function to execute them.
  • Tests and regular text can appear together. The regular texts are ignored by doctest.