Writing Simple Tests

Set up

Create a folder for this lecture:

mkdir python-testing
cd python-testing-tutorial
git init

For this workshop, build the following Dockerfile, and label it ‘testing’. Pip is a package manager for Python; using it, we can install Python libraries. We’ll install the following:

from ubuntu:18.04

RUN apt-get update && apt-get install -y python3 python3-pip

RUN pip install numpy matplotlib

WORKDIR /app

Starting off simply: Squaring a number (a pathological example…)

def squared(x):
    return x*x

Consider just squaring a number. This is so simple, it wouldn’t really be tested in a real piece of software, but we’ll show it here just as an example of how to do it. What could we check?

• Do we get the correct answer for positive numbers?
• Do we get the correct answer for negative numbers?
• Do the input arguments make sense?

It’s clear that all of these would be sensible, but considering it carefully, it shows immediately the difficulty of actually applying testing in practice. There are an infinite amount of numerical input arguments - we cannot reasonably test them all.

How can we therefore reduce it down to a more manageable set of tests? We by necessity have to restrict our tests to a subset of all of the possible outcomes, and have to exercise some thought.

In Python, we can write tests using the Pytest framework. Pytest goes through all of the files in a directory with a “.py” file extension, and looks for normal Python functions which have names that start with “test”. It then runs all of these functions in sequence.

In general, it’s a good idea to write tests in a separate file to the function implementation.

So, to keep track of what we’re doing, we’ll create a new folder and a new Git repository:

In this folder, we’ll create ‘functions.py’ and ‘test_functions.py’

In functions.py add the function from above.

Now, in test_functions.py, we’ll write a test. As mentioned before, you write a normal Python function. However, we use something that you may not have seen before - the assert statement. Asserts just check that something is true; for example:

x = 2
assert x**2 == 4

If you need to, you can provide an error message if an assertion fails:

assert 2 == 1, "The number 2 does not equal 1..."

Add the following function to test_functions.py:

def test_square_positive_integers():
    assert square(1) == 1
    # Note the mistake here! We'll leave this here to
    # show what happens when a test fails.
    assert square(2) == 8
    assert square(4) == 16
    assert square(100) == 10000

Now, using our Docker knowledge from before, we’re going to run this test with Python from a container. Create a Dockerfile with the following contents:

from ubuntu:18.04

RUN apt-get update
RUN apt-get install -y python3
RUN pip3 install pytest

WORKDIR /io

Here, we’re using Pip, the Python package installer to install the Python package pytest.

Build the container:

docker build . -t python-testing

Now, we can run the tests with:

docker run -v$(pwd):/io python-testing pytest -v .

You should see output something like this:

============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.9.2, py-1.7.0, pluggy-0.8.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /io, inifile:
collecting ... collected 1 item

test_functions.py::test_square_positive_integers FAILED                  [100%]

=================================== FAILURES ===================================
________________________ test_square_positive_integers _________________________

    def test_square_positive_integers():
        assert square(1) == 1
        # Note the mistake here! We'll leave this here to
        # show what happens when a test fails.
>       assert square(2) == 8
E       assert 4 == 8
E        +  where 4 = square(2)

test_functions.py:10: AssertionError
=========================== 1 failed in 0.16 seconds ===========================

Note now, that we can see that a test has failed where we introduced the error in the test. We can now correct the test so that it’s expecting the correct answer:

def test_square_positive_integers():
    assert square(1) == 1
    assert square(2) == 4
    assert square(4) == 16
    assert square(100) == 10000

Now if we rerun pytest, we see something different:

============================= test session starts ==============================
platform linux -- Python 3.6.6, pytest-3.9.2, py-1.7.0, pluggy-0.8.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /io, inifile:
collecting ... collected 1 item

test_functions.py::test_square_positive_integers PASSED                  [100%]

=========================== 1 passed in 0.21 seconds ===========================

As many tests as are necessary can be written.

def normalise(v):

>>> import numpy as np


>>> x = np.array([0.0, 1.0])
# We can perform element-wise operations on NumPy arrays:
>>> x / 2
array([0.0, 0.5])

Exceptions

Most programming languages have the concept of exceptions. An exception is just a way of dealing erroneous conditions which happen while a programme is running which require special handling. In Python doing this in your own code is really straightforward. For example, when calculating the Coulomb potential in 1-D, we need to make sure that if the input distance is zero, the function raises an error, because the input argument is invalid. We can do this like:

def CoulombPotential(r):
    if (r == 0):
        raise ValueError("r cannot equal zero.")
    return 1 / abs(r)

Now, when we run this code, if 0 is passed as an input argument:

>>> CoulombPotential(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in CoulombPotential
ValueError: r cannot equal zero.

Debug Mode

Many people do not know that the Python interpreter runs in ‘debug’ mode by default. When it is disabled, by running Python with the ‘-O’ flag, all asserts in code are skipped, and the flag ‘debug’ is set to True. Utilising this can be useful when you want to check code for correctness, but know that some checks you are running can be costly in performance. It can also be used to provide more

def square(x):
  if __debug__:
    print("We're in debug mode!")
  assert type(x) in [int, float, complex], "Input argument x is not of a numeric type int, complex or float"
  return x*x

In debug mode, if we run the function, we get the following output:

>>> square(2)
We're in debug mode!
4
>>> square('a')
We're in debug mode!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in square
AssertionError: Argument is not of a numeric type

When we instead run ‘python3 -O’, we see:

>>> square(2)
4
>>> square('a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in square
TypeError: can't multiply sequence by non-int of type 'str'

We can see that the print statement is completely skipped, and instead of our helpful error message which resulted from our check, we get Python’s less helpful one.

void multiply_vector(const std::vector<double> a,
                     double b,
                     std::vector<double> &c) {

  for(int i = 0; i < a.size(); i++) {
    c[i] = a[i] * b;
    #ifdef MYPROJECT_DEBUG
        std::cout << "c[" << i << "] = " << c[i] << std::endl;
    #endif
  }
}

g++ file.cpp -DMYPROJECT_DEBUG -c

Exercise 2: Lennard-Jones 6-12 potential

The Lennard-Jones potential is given by $$ V\left(r\right) = \epsilon \left[\left(\frac{r_m}{r}\right)^{12} - 2\left(\frac{r_m}{r}\right)^6\right]$$

• Write a function that takes three numbers - $$\epsilon$$, $$r_m$$ and $$r$$, and returns this potential.
• Write a test for this function which checks values against known quantities.
• Write a test that checks that the function throws a ValueError if numerical arguments are not passed.