# Validate Your Network with pytest - Part 1 - Setup

Published: 2021-07-30
Author: Tom Ammon

## What is pytest?

Understanding the operational state of a network is a challenging problem. Understanding the operational readiness of the software that runs the control plane of your network is even more challenging. As our networks grow more complex and more mission-critical, we are being forced to find new ways to verify that they are operating in their intended state. The pytest framework can give us that understanding. pytest “makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries." With the help of a few plugins, we can use pytest to measure the operational state of a network against our own arbitrary criteria. This blog series will give you a running start at using pytest to test your network’s operational state.

## Multipart Series

• Part 1 - Setup (You are Here)
• Part 2 - Kick the Tires
• Part 3 - Add Support for Vendor Devices
• Part 4 - Scale Your Approach

## Lab Setup

This section will focus on the implementation details of the lab I am using to demonstrate the concepts in the rest of the series. If you’re not planning to build out your own version of this environment, you can skip this section and go straight to pytest Hello World - there is nothing particularly applicable to pytest in the lab setup, it’s just giving us some virtual devices to test against.

For all of the examples in this series I am launching pytest from a desktop PC running Ubuntu 20. The devices under test (DUTs) are Debian 10 VMs and Cisco CSR1000v VMs, running on a Debian 10 KVM hypervisor. Here’s how things are connected:

### Topology

A few notes about the topology:

• Debian VMs are running debian 10 (buster) with the FRRouting suite installed.
• links between VMs inside the hypervisor (192.168.1.0/24 above) are accomplished using libvirt’s UDP tunnelling mechanism. This is a “virtual wire” that looks to the VM like a physical cable that’s plugged in.
• Management interfaces on all of the VMs are connected to a linux bridge (br2012), and the hypervisor’s physical NIC is also bridged to br2012, giving us ssh access from our development workstation. This is the path that the pytest connections will travel to connect to the individual VMs.
• libvirt XML definitions (dumped directly from my hypervisor, with no cleanup):

### Development environment

We’ll need to install a few things on our workstation to get our development environment ready:

• pytest itself
• the testinfra pytest plugin - this plugin allows you to run pytest against a remote device

We’ll use a python virtual environment to keep our work contained. I use pyenv with virtualenv. If you’re not familiar with pyenv, here’s a great tutorial.

Inside your virtual environment, install python packages:

$pip install pytest pytest-testinfra paramiko Collecting pytest Using cached pytest-6.2.4-py3-none-any.whl (280 kB) Collecting pytest-testinfra Using cached pytest_testinfra-6.4.0-py3-none-any.whl (71 kB) Collecting paramiko Using cached paramiko-2.7.2-py2.py3-none-any.whl (206 kB) Collecting packaging Using cached packaging-21.0-py3-none-any.whl (40 kB) Collecting iniconfig Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB) Collecting py>=1.8.2 Using cached py-1.10.0-py2.py3-none-any.whl (97 kB) Collecting pluggy<1.0.0a1,>=0.12 Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB) Collecting toml Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting attrs>=19.2.0 Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB) Collecting bcrypt>=3.1.3 Using cached bcrypt-3.2.0-cp36-abi3-manylinux2010_x86_64.whl (63 kB) Collecting pynacl>=1.0.1 Using cached PyNaCl-1.4.0-cp35-abi3-manylinux1_x86_64.whl (961 kB) Collecting cryptography>=2.5 Using cached cryptography-3.4.7-cp36-abi3-manylinux2014_x86_64.whl (3.2 MB) Collecting pyparsing>=2.0.2 Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting cffi>=1.1 Using cached cffi-1.14.6-cp39-cp39-manylinux1_x86_64.whl (405 kB) Collecting six>=1.4.1 Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting pycparser Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB) Installing collected packages: pyparsing, packaging, iniconfig, py, pluggy, toml, attrs, pytest, pytest-testinfra, pycparser, cffi, six, bcrypt, pynacl, cryptography, paramiko Successfully installed attrs-21.2.0 bcrypt-3.2.0 cffi-1.14.6 cryptography-3.4.7 iniconfig-1.1.1 packaging-21.0 paramiko-2.7.2 pluggy-0.13.1 py-1.10.0 pycparser-2.20 pynacl-1.4.0 pyparsing-2.4.7 pytest-6.2.4 pytest-testinfra-6.4.0 six-1.16.0 toml-0.10.2  We won’t be using pytest-testinfra or paramiko right away, but we will need them later on. Now our development environment is ready to go. ## pytest Hello World With our environment set up, we can finally get down to writing code. Let’s write a single test, execute it, and discuss what happened. I’m stealing part of this from the excellent pytest docs. You’ll also want to become familiar with how assert works in python, here’s a quick reference.  1 2 3 4 5 6  # contents of test_first.py print("this is the start of my test") def test_myfirsttest(): assert 5 == 10  Now, it’s time to invoke pytest at the command line. If you’re following along on your own computer (which I highly recommend), you’ll also notice that pytest colorizes the output to make it easier to spot the failed tests.   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22  $ pytest test_first.py ================================ test session starts ================================ platform linux -- Python 3.9.4, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 rootdir: /home/tom/local/repos/blogrepos/network-validation-with-pytest plugins: testinfra-6.4.0 collected 1 item test_first.py F [100%] ===================================== FAILURES ====================================== _________________________________ test_myfirsttest __________________________________ def test_myfirsttest(): > assert 5 == 10 E assert 5 == 10 test_first.py:6: AssertionError ============================== short test summary info ============================== FAILED test_first.py::test_myfirsttest - assert 5 == 10 ================================= 1 failed in 0.02s ================================= 

What’s happening here?

pytest found a function prefixed with “test” on line 5 in test_first.py and executed it for us. By default, pytest runs all functions prefixed with “test”. In pytest parlance, this behavior is called “test discovery”, see the docs for more details.

Look at the output on line 16, and you’ll see that pytest is reporting the results for each individual function/test that failed, in this case, that was test_myfirsttest(). Can you see why this test failed?

Line 18 in the output reveals that the test failed with an “AssertionError” - this means that the condition in the test following the assert did not evaluate to True. In this case, 5 is not equal to 10, and this is what produced the AssertionError. Notice that pytest also prints out some of the python code around the part of the test that failed, and even points you to the line in the file with the failing assert: test_first.py:6: AssertionError. All of this will become very valuable as your library of tests grows.

One last thing: Notice that the print statement didn’t execute. This is because pytest is executing only functions prefixed with “test”. This is another example of how test discovery works.

## Wrap Up

We’ve installed pytest and written a very simple Hello World test. You probably noticed that this test isn’t very useful - comparing the value of two integers on your desktop computer is fun (for some definition of “fun”), but it won’t help you validate the current running state of your network. In the next article, we’ll test some conditions on an actual remote linux router, including the status of a BGP peer - and we’ll see how pytest reacts when that BGP session goes down.

## References

Tags: pytest, python