(c) 2017 Justin Bois and Davi Ortega. This work is licensed under a Creative Commons Attribution License CC-BY 4.0. All code contained herein is licensed under an MIT license.
This tutorial was generated from a Jupyter notebook. You can download the notebook here.
# py.test gives the testing functionality
import pytest
Test-driven development, or TDD, is a paradigm for developing software. The idea is that a programmer thinks about a design specification for a bit of code, usually a function. I.e., she lays out what the input and output should be. She then writes a test (that will fail) for the bit of code. She then writes or updates the code to pass the test. She does this incrementally as she builds her code. Let's try this by example.
We will write a function that computes the number of negatively charged residues in a protein. In other words, we count up the number of glutamate (E) and aspartate (D) residues.
We'll call the function n_neg(), and will just make an empty function for now as a placeholder.
def n_neg(seq):
"""Number of negative residues a protein sequence"""
# Do nothing for now
pass
Now, we'll write a very simple test. It is just a conditional expression.
n_neg('E') == 1
We failed the test! But before we focus on the test failure, let's think about what we just did. We defined the prototype for the function. We know we want it to take in a sequence (a string) and return an integer. So, in building the test, we have designed the interface for the function.
Back to the test failure. We now have a test we would like our function to pass, and we will now revisit the function to write it so that it will pass the test.
def n_neg(seq):
"""Number of negative residues a protein sequence"""
# Count E's and D's, since these are the negative residues
return seq.count('E') + seq.count('D')
We'll try out test again.
n_neg('E') == 1
Hurray! We passed our first test. Now, lets write some more test.
print(n_neg('E') == 1)
print(n_neg('D') == 1)
print(n_neg('') == 0)
print(n_neg('ACKLWTTAE') == 1)
print(n_neg('DDDDEEEE') == 8)
Our function appears to be working well. But let's think carefully about how we could break it. What if we had lowercase letters? I.e., what would we want
n_neg('acklwttae')
to return? Do we allow lower case? This is an example where coming up with tests is how we define the interface. We weren't done designing it at the first pass!
Let's say we want to allow lower case symbols. So, before we mess with our function, let's write a test!
n_neg('acklwttae') == 1
We failed, as expected. Now, back to the function. We will add a line to convert the input sequence to uppercase.
def n_neg(seq):
"""Number of negative residues a protein sequence"""
# Convert sequence to upper case
seq = seq.upper()
# Count E's and D's, since these are the negative residues
return seq.count('E') + seq.count('D')
We need to run ALL of our tests again. We have to make sure everything passes.
print(n_neg('E') == 1)
print(n_neg('D') == 1)
print(n_neg('') == 0)
print(n_neg('ACKLWTTAE') == 1)
print(n_neg('DDDDEEEE') == 8)
print(n_neg('acklwttae') == 1)
Great! This works now.
You can see how the cycle proceeds. Right now, we might be happy with our function, but as we use it in whatever context we are working in, use cases we have not thought of might creep up. For everything that happens, or there is a bug you find, write another test that covers it. Importantly, any time you update your code, you need to run all of your tests!
In our example, we used a bunch of print statements to check our tests. Conveniently, Python have a built-in way to do your tests using the assert keyword. For example, our first test using assert is as follows.
assert n_neg('E') == 1
This ran without issue. Now, let's try asserting something we know will fail.
assert n_neg('E') == 2
We get an AssertionError, indicating that our assertion failed. We can even append the assert statement with a comment describing the error.
assert n_neg('E') == 2, 'Failed on sequence of length 1'
So, we see the basic syntax of assert statements. After assert, we have a conditional expression that evaluates to True or False. If it evaluates False, an AssertionError is raised, meaning that the test was failed. Optionally, the conditional expression can be followed with a comma and a string that describes how it failed. So, we could write all of our tests together as a series of assertions. Actually, it would be best to write a function that does the testing.
def test_n_neg():
"""Perform unit tests on n_neg."""
assert n_neg('E') == 1
assert n_neg('D') == 1
assert n_neg('') == 0
assert n_neg('ACKLWTTAE') == 1
assert n_neg('DDDDEEEE') == 8
assert n_neg('acklwttae') == 1
# Run all the tests
test_n_neg()
Excellent! Everything passed!
It might be a little underwhelming that Python exits silently when all our tests pass. Fortunately, someone else felt that way too and implemented a testing tool that is more into positive reinforcement.
The pytest (a.k.a. py.test) module comes with a standard Anaconda installation and is useful tool for automating your testing. It gives detailed feedback on your tests. You can read its documentation here.
The unittest module from the standard library and nose are two other major testing packages for Python. All three are in common usage. We use pytest here because I think it is the easiest to use and understand.
Pytest is not only a module but also a command line application that searches for tests in your code, runs them and let you know if they fail, and if they pass; finally some positive reinforcement.
To take the most advantage of pytest, we should take a step back, get out of the IPython environment and write the functions we have been working with in this lesson to a .py file. We'll call it seq_features.py. Its content:
def n_neg(seq):
"""Number of negative residues a protein sequence"""
# Convert sequence to upper case
seq = seq.upper()
# Count E's and D's, since these are the negative residues
return seq.count('E') + seq.count('D')
def test_n_neg():
"""Perform unit tests on n_neg."""
assert n_neg('E') == 1
assert n_neg('D') == 1
assert n_neg('') == 0
assert n_neg('ACKLWTTAE') == 1
assert n_neg('DDDDEEEE') == 8
assert n_neg('acklwttae') == 1
Now, pytest makes it easy to verify if all these tests pass or not:
$ pytest seq_features.py
========================== test session starts ==========================
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /Users/Justin/git/programming_bootcamp/2017/lessons, inifile:
collected 1 items
seq_features.py .
======================= 1 passed in 0.00 seconds ========================Hint: try the option -v for even more sugar.
Pytest is such a smart application that you don't even need to tell it explicitly which file it should look at. By default, pytest will search for files starting with test_ or tests_ and ending with .py in the whole directory tree.
It is a good idea to separate your test suite from your code. So let's make another file, named test_seq_features.py and place just the function with the assert statements. You can delete them now from the seq_features.py.
You directory should have now these two files:
seq_features.py
def n_neg(seq):
"""Number of negative residues a protein sequence"""
# Convert sequence to upper case
seq = seq.upper()
# Count E's and D's, since these are the negative residues
return seq.count('E') + seq.count('D')
and
test_seq_features.py
import seq_features
def test_n_neg():
"""Perform unit tests on seq_features.n_neg."""
assert seq_features.n_neg('E') == 1
assert seq_features.n_neg('D') == 1
assert seq_features.n_neg('') == 0
assert seq_features.n_neg('ACKLWTTAE') == 1
assert seq_features.n_neg('DDDDEEEE') == 8
assert seq_features.n_neg('acklwttae') == 1
Note that because the n_neg() function is in a different file than the tests, we must import the seq_features module. Now you can run the test as:
$ pytest -v
========================== test session starts ==========================
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /Users/Justin/git/programming_bootcamp/2017/lessons, inifile:
collected 1 items
test_seq_features.py .
======================= 1 passed in 0.03 seconds ========================Note that there is no need to pass the argument to pytest. This is particularly useful if you have many types of tests, and different applications. Since it is all about organization, now we can separate those tests into more meaningful functional units:
test_seq_features.py
import seq_features
def test_n_neg_for_single_E_or_D():
"""Perform unit tests on n_neg."""
assert seq_features.n_neg('E') == 1
assert seq_features.n_neg('D') == 1
def test_n_neg_for_empty_sequence():
assert seq_features.n_neg('') == 0
def test_n_neg_for_longer_sequences():
assert seq_features.n_neg('ACKLWTTAE') == 1
assert seq_features.n_neg('DDDDEEEE') == 8
def test_n_neg_for_lower_case_sequences():
assert seq_features.n_neg('acklwttae') == 1
And now we can run the tests again.
$ pytest -v
============================== test session starts ===============================
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /Users/Justin/anaconda/bin/python
cachedir: .cache
rootdir: /Users/Justin/git/programming_bootcamp/2017/lessons, inifile:
collected 4 items
test_seq_features.py::test_n_neg_for_single_E_or_D PASSED
test_seq_features.py::test_n_neg_for_empty_sequence PASSED
test_seq_features.py::test_n_neg_for_longer_sequences PASSED
test_seq_features.py::test_n_neg_for_lower_case_sequences PASSED
============================ 4 passed in 0.03 seconds ============================The obvious thing to do next is to test some other cases. Think: what else could go wrong? What if there is an invalid residue in the sequence? How we expect our code to behave?
These and other semi-existential questions will be addressed in the next lesson.