Doctest is the easiest way of performing automated testing in our code. It makes it possible to document and test code at the same time.
Before we look on how to write doctests, it is important to first understand docstrings in Python. Docstrings or documentation strings are simply triple quoted strings that appear as the first statements of a code unit such as a function, class, modules, e.t.c. A docstring typically describe how the code works, how to use it, and any other helpful information that will make the users understand it better.
We can retrieve the docstring of a particular object using the __doc__
attribute. The builtin help()
function also includes the docstring in the displayed help text.
documenting a function
def circle_area(r):
'''Calculates and returns the area of a circle
whose radius is given.'''
area = 3.14 * r * r
return area
print(circle_area.__doc__, end = '\n\n')
help(circle_area)
In the above example, we defined the circle_area()
function which calculates the area of a circle. The function's docstring is the triple quoted text in its body. We printed out the docstring using the __doc__
attribute as well as with the builtin help()
function.
Basic example of Doctest
Doctests are written as a part of the docstring. The basic idea is to verify that the piece of code works correctly by writing statements and the expected outputs in the format of the interactive shell session.
The tests are then executed using the builtin module similarly called doctest
.
Consider the following basic example:
#foo.py
def circle_area(r):
'''
>>> circle_area(r = 7)
153.86
>>> circle_area(r = 9)
254.34
>>> circle_area(r = 15)
706.5
'''
area = 3.14 * r * r
return area
In the above example, we have a file called foo.py
where we defined the function circle_area()
. Look carefully at the function's docstring, it looks as if its contents have been copied directly from an interactive session. Those are actually valid tests. Each statement beginning with >>>
is a function call and the subsequent line is the expected output.
When we run the tests, Python executes any statement that begins with >>>
and compares its return value with the expected value.
To run the tests:
- Open the shell/cmd and change directory to where the file
foo.py
is. - Run the following command.
-
python -m doctest foo.py -v
- The
doctest
framework will automatically retrieve the tests and execute them.
If you run the above command, you should see an output similar to one shown below:
Trying:
circle_area(r = 7)
Expecting:
153.86
ok
Trying:
circle_area(r = 9)
Expecting:
254.34
ok
Trying:
circle_area(r = 15)
Expecting:
706.5
ok
1 items had no tests:
foo
1 items passed all tests:
3 tests in foo.circle_area
3 tests in 2 items.
3 passed and 0 failed.
Test passed.
The above output indicates that all the tests passed. Consider the following example where some tests fail.
#foo.py
def circle_area(r):
'''
>>> circle_area(r = 7)
150
>>> circle_area(r = 9)
254.34
>>> circle_area(r = 15)
706.5
'''
area = 3.14 * r * r
return area
Running the shell command now will lead to the following output.
Trying:
circle_area(r = 7)
Expecting:
150
**********************************************************************
File "C:\Users\John\Desktop\foo.py", line 3, in foo.circle_area
Failed example:
circle_area(r = 7)
Expected:
150
Got:
153.86
Trying:
circle_area(r = 9)
Expecting:
254.34
ok
Trying:
circle_area(r = 15)
Expecting:
706.5
ok
1 items had no tests:
foo
**********************************************************************
1 items had failures:
1 of 3 in foo.circle_area
3 tests in 2 items.
2 passed and 1 failed.
***Test Failed*** 1 failures.
As shown above one test failed because circle_area(r = 7)
is not equal to the expected value i.e 150
.
The syntax of doctests
As earlier mentioned, doctest
identifies tests by looking for parts of the docstring that look as if they were copied directly from an interactive session.
You have most likely used Python's interactive mode in the shell/cmd. Doctests syntax is just similar to that of text from the interactive session. In fact, you can copy lines directly from the interactive shell and paste them as tests in the docstring and they will work just fine.
The structure of the doctests is as described below:
- Lines starting with
>>>
are sent to the interpreter for execution. - A line starting with
...
is sent as a continuation of the code from the previous line. - Bare lines that do not start with any prefix represents the expected outputs from the previous statement(s).
- If no output is given, doctest assumes that a statement should not have any visible output in the console.
doctest
will ignore any piece of text that is not a part of a tests. This means that we can place regular text in the docstring alongside the tests. Consider the following example:
#foo.py
def circle_area(r):
'''
Calculates and returns the area of a circle
whose radius is given.
>>> circle_area(r = 7)
153.86
>>> circle_area(r = 9)
254.34
>>> circle_area(r = 15)
706.5
this is the end of documentation
'''
area = 3.14 * r * r
return area
When you run the python -m doctest foo.py -v
command with the above script, the tests will be executed and the regular text ignored. The output will look as follows:
Trying:
circle_area(r = 7)
Expecting:
153.86
ok
Trying:
circle_area(r = 9)
Expecting:
254.34
ok
Trying:
circle_area(r = 15)
Expecting:
706.5
ok
1 items had no tests:
foo
1 items passed all tests:
3 tests in foo.circle_area
3 tests in 2 items.
3 passed and 0 failed.
Test passed.
Doctest with statements expecting exceptions
In some circumstances we may expect a piece of code to raise an exception. And we may want to write tests to verify that the exception is raised in such cases.
def circle_area(r):
if r < 0 :
raise ValueError(f'A circle with negative radius is undefined.')
area = 3.14 * r * r
return area
print(circle_area(-5))
Traceback (most recent call last):
File "C:\Users\John\Desktop\foo.py", line 10, in <module>
print(circle_area(-5))
^^^^^^^^^^^^^^^
File "C:\Users\John\Desktop\foo.py", line 5, in circle_area
raise ValueError(f'A circle with a negative radius is undefined.')
ValueError: A circle with a negative radius is undefined.
When we want to test that an exception is raised, it would be inconvenient or even impossible to write the entire expected error message including the tracebacks without copy pasting. Doctest makes it easy to test for exceptions by only requiring us to provide only two things:
- The first line i.e
Traceback (most recent call last):
- This tellsdoctest
that we expect an exception to be raised. - The part after the traceback which indicates which exception was raised.
#foo.py
def circle_area(r):
'''
Calculates and returns the area of a circle
whose radius is given.
>>> circle_area(r = 7)
153.86
>>> circle_area(r = - 5)
Traceback (most recent call last):
ValueError: A circle with a negative radius is undefined.
End of documentation
'''
if r < 0 :
raise ValueError(f'A circle with a negative radius is undefined.')
area = 3.14 * r * r
return area
After running the doctest command, the output will be as shown below:
Trying:
circle_area(r = 7)
Expecting:
153.86
ok
Trying:
circle_area(r = - 5)
Expecting:
Traceback (most recent call last):
ValueError: A circle with a negative radius is undefined.
ok
1 items had no tests:
foo
1 items passed all tests:
2 tests in foo.circle_area
2 tests in 2 items.
2 passed and 0 failed.
Test passed.
Testing classes and methods
When defining a class, we can write its tests and those of its method at the top of the class.
#foo.py
class Area:
'''
a class for calculating areas for common shapes.
>>> a = Area()
#test circle
>>> a.circle(r = 7)
153.86
>>> a.circle(r = 15)
706.5
#test triangle
>>> a.triangle(b = 6, h = 8 )
24.0
>>> a.triangle(b = 3, h = 5)
7.5
#test rectangle
>>> a.rectangle(l = 4, w = 5)
20
>>> a.rectangle(l = 6, w = 7)
42
'''
def circle(self, r):
'''
calculates the area of a circle whose radius is r
'''
area = 3.14 * r * r
return area
def triangle(self, b, h):
'''calculates the area of a triangle'''
area = 0.5 * b * h
return area
def rectangle(self, l, w):
'''calculates the area of a rectangle of the specified length and width'''
area = l * w
return area
If you run the doctest command the output will be as shown below:
Trying:
a = Area()
Expecting nothing
ok
Trying:
a.circle(r = 7)
Expecting:
153.86
ok
Trying:
a.circle(r = 15)
Expecting:
706.5
ok
Trying:
a.triangle(b = 6, h = 8 )
Expecting:
24.0
ok
Trying:
a.triangle(b = 3, h = 5)
Expecting:
7.5
ok
Trying:
a.rectangle(l = 4, w = 5)
Expecting:
20
ok
Trying:
a.rectangle(l = 6, w = 7)
Expecting:
42
ok
4 items had no tests:
foo
foo.Area.circle
foo.Area.rectangle
foo.Area.triangle
1 items passed all tests:
7 tests in foo.Area
7 tests in 5 items.
7 passed and 0 failed.
Test passed.
Doctests in external file
If the tests are many and we do not want to clutter the documentation, we can place the tests or some of them in an external file. We can then execute the tests from the main file.
The doctest
module defines the testfile()
function which tests an external file from the calling script's scope. Unlike in the other examples where we were running the tests from the command line, with testsfile()
the tests are executed programmatically.
We will create another file called tests.txt where we will place the tests. The working directory should now have a structure as one shown below:
dir
├── foo.py
└── tests.txt
Let us begin by writing the tests.
#tests.txt
>>> a = Area()
#test circle
>>> a.circle(r = 7)
153.86
>>> a.circle(r = 15)
706.5
#test triangle
>>> a.triangle(b = 6, h = 8 )
24.0
>>> a.triangle(b = 3, h = 5)
7.5
#test rectangle
>>> a.rectangle(l = 4, w = 5)
20
>>> a.rectangle(l = 6, w = 7)
42
We have written the tests for the Area
class in the tests.txt
file. Let us now define the class in the foo.py
file and execute the tests.
#foo.py
class Area:
'''A class for calculating areas of common shapes.'''
def circle(self, r):
'''calculates the area of a circle whose radius is r'''
area = 3.14 * r * r
return area
def triangle(self, b, h):
'''calculates the area of a triangle'''
area = 0.5 * b * h
return area
def rectangle(self, l, w):
'''calculates the area of a rectangle of the specified length and width'''
area = l * w
return area
if __name__ == '__main__':
doctest.testfile('tests.txt', globs = globals(), verbose = True)
Trying:
a = Area()
Expecting nothing
ok
Trying:
a.circle(r = 7)
Expecting:
153.86
ok
Trying:
a.circle(r = 15)
Expecting:
706.5
ok
Trying:
a.triangle(b = 6, h = 8 )
Expecting:
24.0
ok
Trying:
a.triangle(b = 3, h = 5)
Expecting:
7.5
ok
Trying:
a.rectangle(l = 4, w = 5)
Expecting:
20
ok
Trying:
a.rectangle(l = 6, w = 7)
Expecting:
42
ok
1 items passed all tests:
7 tests in test.txt
7 tests in 1 items.
7 passed and 0 failed.
Test passed.
In the above example, we used the testfile()
function to execute the tests defined in tests.txt
file from foo.py
. You can call the builtin help()
function with the testfile
function to see more on its arguments and usage.
import doctest
help(doctest.testfile)
Conclusion:
- Doctest allows us to document and test a piece of code simultaneously.
- The tests are written in the docstring of the target objects.
- The tests should look just as how they would appear in an interactive session.
- To execute the tests we call the
doctest
module from shell/cmd with the following command. -
python -m doctest <name of the file> <args>
- After running the above command,
doctest
automatically looks for the doctests and executes them. - We can also put the tests in an external file then use the
doctest.testfile()
function to execute them. - Tests and regular text can appear together. The regular texts are ignored by
doctest
.