[Python] Testing

The test is the one of most important parts of development.

pytest and unittest are generally used for testing in Python.

Example of unittest Module

Let's use unittest module.

This is a built-in module, so you just use it.

It is highly recommended as follows:

• The name of the test module starts with test.
• Test class inherits from unittest.TestCase.
• The name of the test method starts with test.
• Test methods have at least one assert.

from unittest import TestCase

class CalcTest(TestCase):
    def test_add(self):
        self.assertEqual(1 + 2, 3)
        
    def test_add_with_message(self):
        self.assertEqual(2 + 3, 5, msg="This message is printed when this test fails")

    def test_multiply(self):
        self.assertEqual(3 * 4, 12)

assertEqual() method is provided by TestCase class.

The first parameter is a value to test and the second parameter is a value to expect. 

 

You can also use other asserts.

• assertNotEqual(a, b)
• assertTrue(x)
• asesrtFalse(x)
• assertIsNone(x)
• assertIsNotNone(x)
• assertIs(a, b)
• assertIsNot(a, b)
• assertIn(a, b)
• assertNotIn(a, b)
• assertIsInstance(a, b)
• assertIsInstance(a, b)

 

Try it!

$ python -m unittest <TEST_FILE_NAME>

If you put all the tests inside the tests/ folder, you can omit the TEST_FILE_NAME.

$ python -m unittest

If you want to watch the detailed information, you can use the verbose option (-v).

$ python -m unittest -v

You can run the specific test only instead of the whole test.

Just add the name of the module, class, or method connected by a dot (.).

# Example
$ python -m unittest test_module
$ python -m unittest test_module.test_class
$ python -m unittest test_module.test_class.test_method

You can also run a unit test via code.

unittest.main()

Example of pytest Module

pytest module is a third-party module.

Therefore, it should be installed before use.

$ pip install pytest

import pytest

def test_add():
    assert (1 + 2) == 3

def test_multiply():
    assert (2 * 3) == 6

You don't need a class, just assert test code.

 

Try it!

$ pytest <TEST_FILE_NAME>

If you want to watch the detailed information, you can use the verbose option (-v).

$ pytest -v <TEST_FILE_NAME>

You can run the specific test only instead of the whole test.

$ pytest <TEST_FILE_NAME> -k <TEST_NAME>

Test Fixture in unittest Module

class TestSuite(TestCase):
    def setUp(self):
        pass

    def tearDown(self):
        pass

setUp() method is called before every single test is executed, and tearDown() method is called after every single test is executed.

Test Fixture in pytest Module

There are several ways to use test fixtures in pytest.

def setup_function():
    pass
    
def teardown_function():
    pass

@pytest.fixture
def my_setup():
    print("This is for setup")
    yield
    print("This is for teardown")

def test_add(my_setup):
    assert (1 + 2) == 3

You can use it either way.

You are also able to use the fixture's return value.

@pytest.fixture
def my_fixture():
    data = 7
    yield data
    data = 0
    
def test_add(my_fixture):
    assert (my_fixture + 2) == 9

Instead of specifying in each test, you can also automatically use it for all tests.

@pytest.fixture(autouse=True)
def my_fixture():
    pass

You can pass the parameter with a test fixture.

@pytest.fixture(params=["param1", "param2"])
def my_fixture(request):
    data = request.param
    yield data
    data = None
    
def test_list(my_fixture):
    if my_fixture == "param1":
        assert "param1" in my_fixture
    elif my_fixture == "param2":
        assert "param2" in my_fixture

The test_list is called twice because the number of parameters is two.

You can easily multiple tests through this way.

Multiple Tests with Parameter

You can use subTest() method if you want to do the same test several times by changing arguments.

def testAdd(self):
    params = ((3, {"a": 1, "b": 2}), (5, {"a": 2, "b": 3}), (7, {"a": 3, "b": 4}))
    for expected, param in params:
        with self.subTest(**param):
            actual = param["a"] + param["b"]
            self.assertEqual(actual, expected)

If you use subTest() method, all subtests are executed even if there is a failure in the middle.

Test Skip in unittest Module

unittest Module provides a decorator to skip tests.

@unittest.skip("This test will be skipped")
def testSkip(self):
    pass

@unittest.skipIf(2 > 1, "This test will be skipped")
def testSkipIf(self):
    pass

Test Skip in pytest Module

@pytest.mark.skip("This test will be skipped")
def test_false():
    assert False

@pytest.mark.skipif(3 > 1, reason="This test is not avaliable")
def test_true():
    assert True

def test_skip():
    if True:
        pytest.skip("This test will be skipped")

You can use a decorator or skip() method for skipping some tests.

Test Grouping in pytest Module

First, indicate the group name you want for the test.

@pytest.mark.multiply
def test_multiply1():
    assert (2 * 3) == 6

@pytest.mark.multiply
def test_multiply2():
    assert (3 * 4) == 12

I named the group as multiply in the example above.

Next, register the group name into the configuration file.

; pytest.ini

[pytest]
markers =
    multiply

Now, run the tests!

$ pytest <FILE_NAME> -m <GROUP_NAME>

pytest with Multi Processor

If you want to use a multiprocessor when tests run, pytest-xdist module is needed.

$ pip install pytest-xdist

Now, the numprocesses option (-n) is available.

$ pytest -n <NUMBER_OF_PROCESSORS_TO_USE> <TEST_FILE_NAME>

Mock in unittest Module

In cases where the test environment cannot be configured the same, the test is often performed using a mock object.

from unittest import TestCase, mock

m = mock.Mock()
m.my_attribute = 7
m.my_method.return_value = 3
m.side_effect = lambda x: x % 2

class CalcTest(TestCase):
    def test_add(self):
        self.assertEqual(m.my_attribute + 2, 9)
        m.my_method.assert_not_called()

    def test_multiply(self):
        self.assertEqual(m.my_method() * 3, 9)
        m.my_method.assert_called_once()

    def test_mock(self):
        self.assertEqual(m(2), 0)
        m.assert_called_with(2)
        m.assert_called_once_with(2)

Let's take a look at the mock object.

You can set the return_value at the creating time.

m = mock.Mock(return_value=3)
print(m())	# 3

You can put the side_effect with function, iterable, or exception.

m = mock.Mock(side_effect=lambda x: x % 2)
print(m(3))  # 1

m.side_effect = [1, 2, 3]
print(m())  # 1
print(m())  # 2
print(m())  # 3

m.side_effect = ValueError("Error")
print(m())  # ValueError: Error

patch() method replaces the desired object with a mock object.

# calc.py

def add(x, y: int) -> int:
    return x + y

# test_patch.py

from unittest import TestCase, mock

import calc

class CalcTest(TestCase):
    def test_patch1(self):
        with mock.patch("calc.add") as mock_add:
            mock_add.return_value = 3
            self.assertEqual(calc.add(1, 2), 3)

    @mock.patch("calc.add")
    def test_patch2(self, mock_add):
        mock_add.return_value = 3
        self.assertEqual(calc.add(1, 2), 3)

Either way works well.

Coverage in unittest Module

coverage module helps you to report coverage.

$ pip install coverage

Now, run the tests using the coverage module and report it.

$ coverage run -m unittest <TEST_FILE_NAME>
$ coverage report -m

Coverage in pytest Module

If you want to report the test coverage, pytest-cov is needed.

$ pip install pytest-cov

Now, you can use the coverage option (--cov).

$ pytest --cov=<MODULE_NAME> <TEST_FILE_NAME>