1+ :lastproofread: 2025-12-02
2+
13.. _testing :
24
35#######
46Testing
57#######
68
7- One of the major advantages introduced in VyOS 1.3 is an automated test
8- framework. When assembling an ISO image multiple things can go wrong badly and
9- publishing a faulty ISO makes no sense. The user is disappointed by the quality
10- of the image and the developers get flodded with bug reports over and over
11- again.
9+ One of the major features introduced in VyOS 1.3 is an automated test
10+ framework. When you assemble an ISO image, several things can go wrong.
11+ VyOS uses this framework to detect issues before they cause downstream problems.
1212
13- As the VyOS documentation is not only for users but also for the developers -
14- and we keep no secret documentation - this section describes how the automated
15- testing works.
13+ This section describes how the automated testing process at VyOS works.
1614
1715Jenkins CI
1816==========
1917
20- Our `VyOS CI `_ system is based on Jenkins and builds all our required packages
21- for VyOS 1.2 to 1.4. In addition to the package build, there is the vyos-build
22- Job which builds and tests the VyOS ISO image which is published after a
23- successful test drive .
18+ The `VyOS CI `_ system is based on Jenkins. It builds all required packages
19+ for VyOS 1.2 to 1.4. In addition to the package build, there is the
20+ `` vyos-build `` job, which builds and tests the VyOS ISO image.
21+ The image is published after a successful test run .
2422
25- We differentiate in two independent tests, which are both run in parallel by
26- two separate QEmu instances which are launched via ``make test `` and ``make
27- testc `` from within the vyos-build _ repository.
23+ VyOS runs two independent tests in parallel using separate QEMU instances.
24+ These are launched via ``make test `` and ``make testc `` from within the
25+ vyos-build _ repository.
2826
2927Smoketests
3028==========
3129
32- Smoketests executes predefined VyOS CLI commands and checks if the desired
30+ Smoketests execute predefined VyOS CLI commands and check if the desired
3331daemon/service configuration is rendert - that is how to put it "short".
3432
35- When and ISO image is assembled by the `VyOS CI `_, the ``BUILD_SMOKETEST ``
36- parameter is enabled by default, which will extend the ISO configuration line
33+ When an ISO image is assembled by the `VyOS CI `_, the ``BUILD_SMOKETEST ``
34+ parameter is enabled by default. This extends the ISO configuration line
3735with the following packages:
3836
3937.. code-block :: python
@@ -42,29 +40,29 @@ with the following packages:
4240 if (params.BUILD_SMOKETESTS )
4341 CUSTOM_PACKAGES = ' --custom-package vyos-1x-smoketest'
4442
45- So if you plan to build your own custom ISO image and want to make use of our
43+ If you plan to build your own custom ISO image and want to use VyOS's
4644smoketests, ensure that you have the `vyos-1x-smoketest ` package installed.
4745
48- The ``make test `` command from the vyos-build _ repository will launch a new
49- QEmu instance and the ISO image is first installed to the virtual harddisk .
46+ The ``make test `` command from the vyos-build _ repository launches a new
47+ QEMU instance, and the ISO image is first installed to the virtual hard disk .
5048
51- After its first boot into the newly installed system the main Smoketest script
52- is executed, it can be found here: `/usr/bin/vyos-smoketest `
49+ After the first boot into the newly installed system, the main Smoketest script
50+ is executed. It can be found at `/usr/bin/vyos-smoketest `.
5351
5452The script only searches for executable "test-cases" under
5553``/usr/libexec/vyos/tests/smoke/cli/ `` and executes them one by one.
5654
57- .. note :: As Smoketests will alter the system configuration and you are logged
58- in remote you may loose your connection to the system.
55+ .. note :: Smoketests will alter the system configuration. If you are logged
56+ in remotely, you may lose your connection to the system.
5957
60- .. note :: To enable smoketest debugging (print of the CLI set commands used)
61- you can run: ``touch /tmp/vyos.smoketest.debug ``.
58+ .. note :: To enable smoketest debugging (print the CLI set commands used),
59+ run: ``touch /tmp/vyos.smoketest.debug ``.
6260
6361Manual Smoketest Run
6462--------------------
6563
66- On the other hand - as each test is contain in its own file - one can always
67- execute a single Smoketest by hand by simply running the Python test scripts .
64+ Each test is contained in its own file, so you can execute a single Smoketest
65+ manually by running the Python test script .
6866
6967Example:
7068
@@ -93,12 +91,11 @@ Example:
9391 Interface based tests
9492---------------------
9593
96- Our smoketests not only test daemons and serives, but also check if what we
97- configure for an interface works. Thus there is a common base classed named:
98- ``base_interfaces_test.py `` which holds all the common code that an interface
99- supports and is tested.
94+ Our smoketests not only test daemons and services, but also check if interface
95+ configuration works as expected. There is a common base class named
96+ ``base_interfaces_test.py `` that holds all the common code for interface tests.
10097
101- Those common tests consists out of:
98+ These common tests consist of:
10299
103100* Add one or more IP addresses
104101* DHCP client and DHCPv6 prefix delegation
@@ -109,12 +106,12 @@ Those common tests consists out of:
109106* VLANs (QinQ and regular 802.1q)
110107* ...
111108
112- .. note :: When you are working on interface configuration and you also want to
113- test if the Smoketests pass you would normally loose the remote SSH connection
114- to your :abbr: `DUT ( Device Under Test ) `. To handle this issue , some of the
115- interface based tests can be called with an environment variable beforehand
116- to limit the number of interfaces used in the test. By default all interface
117- e.g. all Ethernet interfaces are used.
109+ .. note :: When you are working on interface configuration and want to test
110+ if the Smoketests pass, you would normally lose the remote SSH connection
111+ to your :abbr: `DUT ( Device Under Test ) `. To handle this, some interface-based
112+ tests can be called with an environment variable beforehand to limit the
113+ number of interfaces used in the test. By default, all interfaces (e.g., all
114+ Ethernet interfaces) are used.
118115
119116.. code-block :: none
120117
@@ -148,31 +145,32 @@ Those common tests consists out of:
148145
149146 OK
150147
151- This will limit the `bond ` interface test to only make use of `eth1 ` and `eth2 `
148+ This will limit the `bond ` interface test to use only `eth1 ` and `eth2 `
152149as member ports.
153150
154151Config Load Tests
155152=================
156153
157- The other part of our tests are called "config load tests". The config load tests
158- will load - one after another - arbitrary configuration files to test if the
159- configuration migration scripts work as designed and that a given set of
160- functionality still can be loaded with a fresh VyOS ISO image.
154+ The other part of our tests are called "config load tests." Config load tests
155+ sequentially load arbitrary configuration files to verify that configuration
156+ migration scripts work as designed and that a given set of functionality can
157+ still be loaded with a fresh VyOS ISO image.
161158
162- The configurations are all derived from production systems and can not only act
163- as a testcase but also as reference if one wants to enable a certain feature.
164- The configurations can be found here:
159+ The configurations are all derived from production systems and can act as
160+ test cases or as references for enabling certain features. The configurations
161+ can be found here:
165162https://github.com/vyos/vyos-1x/tree/current/smoketest/configs
166163
167- The entire test is controlled by the main wrapper script ``/usr/bin/vyos-configtest ``
168- which behaves in the same way as the main smoketest script. It scans the folder
169- for potential configuration files and issues a ``load `` command one after another.
164+ The entire test is controlled by the main wrapper script
165+ ``/usr/bin/vyos-configtest ``.
166+ It behaves in the same way as the main smoketest script. It scans the folder
167+ for potential configuration files and issues a ``load `` command for each file.
170168
171169Manual config load test
172170-----------------------
173171
174- One is not bound to load all configurations one after another but can also load
175- individual test configurations on his own .
172+ You do not have to load all configurations sequentially; you can also load
173+ individual test configurations manually .
176174
177175.. code-block :: none
178176
@@ -202,10 +200,10 @@ individual test configurations on his own.
202200 vyos@vyos# commit
203201 vyos@vyos#
204202
205- .. note :: Some of the configurations have preconditions which need to be met.
206- Those most likely include generation of crypographic keys before the config
207- can be applied - you will get a commit error otherwise . If you are interested
208- how those preconditions are fulfilled check the vyos-build _ repository and
209- the ``scripts/check-qemu-install `` file.
203+ .. note :: Some configurations have preconditions that must be met. These most
204+ likely include generation of cryptographic keys before the config can be
205+ applied; otherwise, you will get a commit error. If you are interested in
206+ how those preconditions are fulfilled, check the vyos-build _ repository and
207+ the ``scripts/check-qemu-install `` file.
210208
211209.. include :: /_include/common-references.txt
0 commit comments