From b6a7edc2c60f7755176a5bb6eaf1ab3d8577a66c Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Sun, 30 Nov 2025 17:32:39 +0900 Subject: [PATCH 01/10] Proofread upstream-packages.rst --- docs/contributing/upstream-packages.rst | 110 ++++++++++++------------ 1 file changed, 56 insertions(+), 54 deletions(-) diff --git a/docs/contributing/upstream-packages.rst b/docs/contributing/upstream-packages.rst index 5c48bbb3d6..64fe85fc5d 100644 --- a/docs/contributing/upstream-packages.rst +++ b/docs/contributing/upstream-packages.rst @@ -1,86 +1,88 @@ +:lastproofread: 2025-11-30 + .. _upstream_packages: -Upstream packages ------------------ +################# +Upstream Packages +################# -Many base system packages are pulled straight from Debian's main and contrib -repositories, but there are exceptions. +Many base system packages are pulled straight from Debian's ``main`` and +``contrib`` repositories, but there are exceptions. -This chapter lists those exceptions and gives you a brief overview what we -have done on those packages. If you only want to build yourself a fresh ISO -you can completely skip this chapter. It may become interesting once you have -a VyOS deep dive. +This page lists those exceptions and briefly describes changes made to +these packages. If you only want to build a fresh ISO image, you can skip +this section. This information may be useful for a deeper dive into VyOS. -vyos-netplug -^^^^^^^^^^^^ +``vyos-netplug`` +---------------- -Due to issues in the upstream version that sometimes set interfaces down, a -modified version is used. +VyOS uses a modified version because the upstream release sometimes causes +network interfaces to go down. -The source is located at https://github.com/vyos/vyos-netplug +Source: https://github.com/vyos/vyos-netplug. -In the future, we may switch to using systemd infrastructure instead. Building -it doesn't require a special procedure. +VyOS may switch to ``systemd`` in the future. Building the package does not +require a special procedure. -keepalived -^^^^^^^^^^ +``keepalived`` +-------------- -Keepalived normally isn't updated to newer feature releases between Debian -versions, so we are building it from source. +``keepalived`` is not updated to newer feature releases between Debian releases. +VyoS builds it from source. -Debian does keep their package in git, but it's upstream tarball imported into -git without its original commit history. To be able to merge new tags in, we -keep a fork of the upstream repository with packaging files imported from -Debian at https://github.com/vyos/keepalived-upstream +Debian maintains the package in git, but the upstream tarball was imported +without its original commit history. To allow merging new tags, we maintain +a fork with packaging files imported from Debian: https://github.com/vyos/keepalived-upstream. -strongswan -^^^^^^^^^^ +``strongswan`` +-------------- -Our StrongSWAN build differs from the upstream: +VyOS's StrongSWAN build differs from upstream: -- strongswan-nm package build is disabled since we don't use NetworkManager -- Patches for DMVPN are merged in +- We disable the ``strongswan-nm`` package build because VyOS does not use + NetworkManager. +- We merged patches for DMVPN. -The source is at https://github.com/vyos/vyos-strongswan +Source: https://github.com/vyos/vyos-strongswan -DMVPN patches are added by this commit: +DMVPN patches were added in this commit: https://github.com/vyos/vyos-strongswan/commit/1cf12b0f2f921bfc51affa3b81226 -Our op mode scripts use the python-vici module, which is not included in -Debian's build, and isn't quite easy to integrate in that build. For this -reason we debianize that module by hand now, using this procedure: +VyOS's op-mode scripts use the ``python-vici`` module, which is not included +in Debian's build and is difficult to integrate. VyOS debianizes the module +manually: -0. Install https://pypi.org/project/stdeb/ -1. `cd vyos-strongswan` -2. `./configure --enable-python-eggs` -3. `cd src/libcharon/plugins/vici/python` -4. `make` -5. `python3 setup.py --command-packages=stdeb.command bdist_deb` +1. Install ``stdeb`` from PyPI (for example: ``pip3 install stdeb``). +2. ``cd vyos-strongswan`` +3. ``./configure --enable-python-eggs`` +4. ``cd src/libcharon/plugins/vici/python`` +5. ``make`` +6. ``python3 setup.py --command-packages=stdeb.command bdist_deb`` -The package ends up in deb_dist dir. +The package is created in the ``deb_dist`` directory. -mdns-repeater -^^^^^^^^^^^^^ +``mdns-repeater`` +----------------- -This package doesn't exist in Debian. A debianized fork is kept at -https://github.com/vyos/mdns-repeater +This package does not exist in Debian. VyOS maintains a debianized fork at +https://github.com/vyos/mdns-repeater. No special build procedure is required. -udp-broadcast-relay -^^^^^^^^^^^^^^^^^^^ +``udp-broadcast-relay`` +----------------------- -This package doesn't exist in Debian. A debianized fork is kept at -https://github.com/vyos/udp-broadcast-relay +This package does not exist in Debian. VyOS maintain a debianized fork at +https://github.com/vyos/udp-broadcast-relay. No special build procedure is required. -hvinfo -^^^^^^ +``hvinfo`` +---------- -A fork with packaging changes for VyOS is kept at https://github.com/vyos/hvinfo +A fork with packaging changes for VyOS is available at https://github.com/vyos/hvinfo. -The original repo is at https://github.com/dmbaturin/hvinfo +The original repository is at https://github.com/dmbaturin/hvinfo. -It's an Ada program and requires GNAT and gprbuild for building, dependencies -are properly specified so just follow debuild's suggestions. +It is an Ada program and requires GNAT and ``gprbuild``. Dependencies are +properly specified; follow the suggestions from ``debuild``. From 4379d62c1eec01b74fbbdeb6be38905e3c92e081 Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Sun, 30 Nov 2025 17:35:08 +0900 Subject: [PATCH 02/10] Fix line length lint errors --- docs/contributing/upstream-packages.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/contributing/upstream-packages.rst b/docs/contributing/upstream-packages.rst index 64fe85fc5d..78fe374512 100644 --- a/docs/contributing/upstream-packages.rst +++ b/docs/contributing/upstream-packages.rst @@ -32,7 +32,8 @@ VyoS builds it from source. Debian maintains the package in git, but the upstream tarball was imported without its original commit history. To allow merging new tags, we maintain -a fork with packaging files imported from Debian: https://github.com/vyos/keepalived-upstream. +a fork with packaging files imported from +Debian: https://github.com/vyos/keepalived-upstream. ``strongswan`` -------------- @@ -80,7 +81,8 @@ No special build procedure is required. ``hvinfo`` ---------- -A fork with packaging changes for VyOS is available at https://github.com/vyos/hvinfo. +A fork with packaging changes for VyOS is available +at https://github.com/vyos/hvinfo. The original repository is at https://github.com/dmbaturin/hvinfo. From cc462ed3f70c42f2ba14aa1c0ba06378839977ca Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Wed, 3 Dec 2025 11:04:16 +0900 Subject: [PATCH 03/10] Initial copyedit for testing.rst --- docs/contributing/testing.rst | 114 +++++++++++++++++----------------- 1 file changed, 56 insertions(+), 58 deletions(-) diff --git a/docs/contributing/testing.rst b/docs/contributing/testing.rst index f5d3db5260..ea9a683b41 100644 --- a/docs/contributing/testing.rst +++ b/docs/contributing/testing.rst @@ -1,39 +1,37 @@ +:lastproofread: 2025-12-02 + .. _testing: ####### Testing ####### -One of the major advantages introduced in VyOS 1.3 is an automated test -framework. When assembling an ISO image multiple things can go wrong badly and -publishing a faulty ISO makes no sense. The user is disappointed by the quality -of the image and the developers get flodded with bug reports over and over -again. +One of the major features introduced in VyOS 1.3 is an automated test +framework. When you assemble an ISO image, several things can go wrong. +VyOS uses this framework to detect issues before they cause downstream problems. -As the VyOS documentation is not only for users but also for the developers - -and we keep no secret documentation - this section describes how the automated -testing works. +This section describes how the automated testing process at VyOS works. Jenkins CI ========== -Our `VyOS CI`_ system is based on Jenkins and builds all our required packages -for VyOS 1.2 to 1.4. In addition to the package build, there is the vyos-build -Job which builds and tests the VyOS ISO image which is published after a -successful test drive. +The `VyOS CI`_ system is based on Jenkins. It builds all required packages +for VyOS 1.2 to 1.4. In addition to the package build, there is the +``vyos-build`` job, which builds and tests the VyOS ISO image. +The image is published after a successful test run. -We differentiate in two independent tests, which are both run in parallel by -two separate QEmu instances which are launched via ``make test`` and ``make -testc`` from within the vyos-build_ repository. +VyOS runs two independent tests in parallel using separate QEMU instances. +These are launched via ``make test`` and ``make testc`` from within the +vyos-build_ repository. Smoketests ========== -Smoketests executes predefined VyOS CLI commands and checks if the desired +Smoketests execute predefined VyOS CLI commands and check if the desired daemon/service configuration is rendert - that is how to put it "short". -When and ISO image is assembled by the `VyOS CI`_, the ``BUILD_SMOKETEST`` -parameter is enabled by default, which will extend the ISO configuration line +When an ISO image is assembled by the `VyOS CI`_, the ``BUILD_SMOKETEST`` +parameter is enabled by default. This extends the ISO configuration line with the following packages: .. code-block:: python @@ -42,29 +40,29 @@ with the following packages: if (params.BUILD_SMOKETESTS) CUSTOM_PACKAGES = '--custom-package vyos-1x-smoketest' -So if you plan to build your own custom ISO image and want to make use of our +If you plan to build your own custom ISO image and want to use VyOS's smoketests, ensure that you have the `vyos-1x-smoketest` package installed. -The ``make test`` command from the vyos-build_ repository will launch a new -QEmu instance and the ISO image is first installed to the virtual harddisk. +The ``make test`` command from the vyos-build_ repository launches a new +QEMU instance, and the ISO image is first installed to the virtual hard disk. -After its first boot into the newly installed system the main Smoketest script -is executed, it can be found here: `/usr/bin/vyos-smoketest` +After the first boot into the newly installed system, the main Smoketest script +is executed. It can be found at `/usr/bin/vyos-smoketest`. The script only searches for executable "test-cases" under ``/usr/libexec/vyos/tests/smoke/cli/`` and executes them one by one. -.. note:: As Smoketests will alter the system configuration and you are logged - in remote you may loose your connection to the system. +.. note:: Smoketests will alter the system configuration. If you are logged + in remotely, you may lose your connection to the system. -.. note:: To enable smoketest debugging (print of the CLI set commands used) - you can run: ``touch /tmp/vyos.smoketest.debug``. +.. note:: To enable smoketest debugging (print the CLI set commands used), + run: ``touch /tmp/vyos.smoketest.debug``. Manual Smoketest Run -------------------- -On the other hand - as each test is contain in its own file - one can always -execute a single Smoketest by hand by simply running the Python test scripts. +Each test is contained in its own file, so you can execute a single Smoketest +manually by running the Python test script. Example: @@ -93,12 +91,11 @@ Example: Interface based tests --------------------- -Our smoketests not only test daemons and serives, but also check if what we -configure for an interface works. Thus there is a common base classed named: -``base_interfaces_test.py`` which holds all the common code that an interface -supports and is tested. +Our smoketests not only test daemons and services, but also check if interface +configuration works as expected. There is a common base class named +``base_interfaces_test.py`` that holds all the common code for interface tests. -Those common tests consists out of: +These common tests consist of: * Add one or more IP addresses * DHCP client and DHCPv6 prefix delegation @@ -109,12 +106,12 @@ Those common tests consists out of: * VLANs (QinQ and regular 802.1q) * ... -.. note:: When you are working on interface configuration and you also want to - test if the Smoketests pass you would normally loose the remote SSH connection - to your :abbr:`DUT (Device Under Test)`. To handle this issue, some of the - interface based tests can be called with an environment variable beforehand - to limit the number of interfaces used in the test. By default all interface - e.g. all Ethernet interfaces are used. +.. note:: When you are working on interface configuration and want to test + if the Smoketests pass, you would normally lose the remote SSH connection + to your :abbr:`DUT (Device Under Test)`. To handle this, some interface-based + tests can be called with an environment variable beforehand to limit the + number of interfaces used in the test. By default, all interfaces (e.g., all + Ethernet interfaces) are used. .. code-block:: none @@ -148,31 +145,32 @@ Those common tests consists out of: OK -This will limit the `bond` interface test to only make use of `eth1` and `eth2` +This will limit the `bond` interface test to use only `eth1` and `eth2` as member ports. Config Load Tests ================= -The other part of our tests are called "config load tests". The config load tests -will load - one after another - arbitrary configuration files to test if the -configuration migration scripts work as designed and that a given set of -functionality still can be loaded with a fresh VyOS ISO image. +The other part of our tests are called "config load tests." Config load tests +sequentially load arbitrary configuration files to verify that configuration +migration scripts work as designed and that a given set of functionality can +still be loaded with a fresh VyOS ISO image. -The configurations are all derived from production systems and can not only act -as a testcase but also as reference if one wants to enable a certain feature. -The configurations can be found here: +The configurations are all derived from production systems and can act as +test cases or as references for enabling certain features. The configurations +can be found here: https://github.com/vyos/vyos-1x/tree/current/smoketest/configs -The entire test is controlled by the main wrapper script ``/usr/bin/vyos-configtest`` -which behaves in the same way as the main smoketest script. It scans the folder -for potential configuration files and issues a ``load`` command one after another. +The entire test is controlled by the main wrapper script +``/usr/bin/vyos-configtest``. +It behaves in the same way as the main smoketest script. It scans the folder +for potential configuration files and issues a ``load`` command for each file. Manual config load test ----------------------- -One is not bound to load all configurations one after another but can also load -individual test configurations on his own. +You do not have to load all configurations sequentially; you can also load +individual test configurations manually. .. code-block:: none @@ -202,10 +200,10 @@ individual test configurations on his own. vyos@vyos# commit vyos@vyos# -.. note:: Some of the configurations have preconditions which need to be met. - Those most likely include generation of crypographic keys before the config - can be applied - you will get a commit error otherwise. If you are interested - how those preconditions are fulfilled check the vyos-build_ repository and - the ``scripts/check-qemu-install`` file. +.. note:: Some configurations have preconditions that must be met. These most + likely include generation of cryptographic keys before the config can be + applied; otherwise, you will get a commit error. If you are interested in + how those preconditions are fulfilled, check the vyos-build_ repository and + the ``scripts/check-qemu-install`` file. .. include:: /_include/common-references.txt From e1aad103dd79710f87fc024c76948e0b8364a894 Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Fri, 5 Dec 2025 21:09:53 +0900 Subject: [PATCH 04/10] Proofread build-vyos.rst --- docs/contributing/build-vyos.rst | 214 +++++++++++++++---------------- docs/contributing/testing.rst | 6 +- 2 files changed, 104 insertions(+), 116 deletions(-) diff --git a/docs/contributing/build-vyos.rst b/docs/contributing/build-vyos.rst index 1b321332fc..003104b272 100644 --- a/docs/contributing/build-vyos.rst +++ b/docs/contributing/build-vyos.rst @@ -1,3 +1,5 @@ +:lastproofread: 2025-12-05 + .. _build: ########## @@ -8,11 +10,9 @@ Build VyOS Prerequisites ************* -There are different ways you can build VyOS. - -Building using a :ref:`build_docker` container, although not the only way, -is the easiest way as all dependencies are managed for you. However, you can -also set up your own build machine and run a :ref:`build_native`. +There are different ways you can build VyOS. Building using a :ref:`build_docker` +container is the easiest way because all dependencies are managed for you. +Alternatively, you can set up your own build machine and run a :ref:`build_native` build. .. note:: Starting with VyOS 1.4, only source code and Debian package repositories of the rolling release (the **current** branch) are publicly available. @@ -31,24 +31,18 @@ This process has been tested on clean installs of Debian Bookworm. Native Build ============ -To build VyOS natively you need a properly configured build host with the -following Debian versions installed: - -- Debian Bookworm +To build VyOS natively, you need a properly configured build host with +Debian Bookworm installed. -To start, clone the repository to your local machine: +To get started, clone the repository to your local machine: .. code-block:: none $ sudo make clean $ sudo ./build-vyos-image --architecture amd64 --build-by "j.randomhacker@vyos.io" generic -For the packages required, you can refer to the ``docker/Dockerfile`` file -in the repository_. The ``./build-vyos-image`` script will also warn you if any -dependencies are missing. - -This will guide you through the process of building a VyOS ISO using Docker. -This process has been tested on clean installs of Bookworm (12). +For required packages, refer to the ``docker/Dockerfile`` file in the repository_. +The ``./build-vyos-image`` script will also warn you if any dependencies are missing. .. _build_docker: @@ -57,8 +51,8 @@ Docker Installing Docker_ and prerequisites: -.. hint:: Due to the updated version of Docker, the following examples may - become invalid. +.. hint:: Docker versions are updated frequently. The following examples may + become outdated. .. code-block:: none @@ -78,29 +72,27 @@ Installing Docker_ and prerequisites: sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -To be able to use Docker_ without ``sudo``, the current non-root user must be -added to the ``docker`` group by calling: ``sudo usermod -aG docker -yourusername``. +To use Docker without ``sudo``, add your current non-root user to the ``docker`` +group: ``sudo usermod -aG docker yourusername``. -.. hint:: Doing so grants privileges equivalent to the ``root`` user! It is - recommended to remove the non-root user from the ``docker`` group after - building the VyOS ISO. See also `Docker as non-root`_. +.. hint:: Adding a user to the ``docker`` group grants privileges equivalent to + ``root``. It is recommended to remove the non-root user from the ``docker`` + group after building the VyOS ISO. See also `Docker as non-root`_. -.. note:: The build process needs to be built on a local file system, building - on SMB or NFS shares will result in the container failing to build properly! - VirtualBox Drive Share is also not an option as block device operations - are not implemented and the drive is always mounted as "nodev" +.. note:: The build process must run on a local file system. Building on SMB or + NFS shares will cause the container to fail. VirtualBox shared folders are + also not supported because block device operations are not implemented. Build Container --------------- The container can be built by hand or by fetching the pre-built one from -DockerHub. Using the pre-built containers from the `VyOS DockerHub -organisation`_ will ensure that the container is always up-to-date. A rebuild -is triggered once the container changes (please note this will take 2-3 hours -after pushing to the vyos-build repository). +DockerHub. It is recommended to use the pre-built containers from the +`VyOS DockerHub`organization_ +The container is built from Docker packages automatically after every commit +to the ``vyos-build`` repository (this process may take 2-3 hours). -.. note: If you are using the pre-built container, it will be automatically +.. note:: If you use the pre-built container, it will be automatically downloaded from DockerHub if it is not found on your local machine when you build the ISO. @@ -125,16 +117,15 @@ The container can also be built directly from source: $ cd vyos-build $ docker build -t vyos/vyos-build:current docker -.. note:: VyOS has switched to Debian (12) Bookworm in its ``current`` branch, - Due to software version updates, it is recommended to use the official +.. note:: VyOS switched to Debian Bookworm (12) in its ``current`` branch. + Due to software version updates, it is recommended to use the official Docker Hub image to build VyOS ISO. Tips and Tricks --------------- -You can create yourself some handy Bash aliases to always launch the latest - -per release train (`current`) - container. Add the following to your -``.bash_aliases`` file: +You can create Bash aliases to easily launch the latest container per release +train (``current``). Add the following to your ``.bash_aliases`` file: .. code-block:: none @@ -147,8 +138,8 @@ per release train (`current`) - container. Add the following to your -e GOSU_UID=$(id -u) -e GOSU_GID=$(id -g) \ vyos/vyos-build:current bash' -Now you are prepared with a new aliase ``vybld`` to spawn -your development containers in your current working directory. +Now you have a new alias ``vybld`` that launches development containers in +your current working directory. .. note:: Some VyOS packages (namely vyos-1x) come with build-time tests which verify some of the internal library calls that they work as expected. Those @@ -165,16 +156,16 @@ your development containers in your current working directory. Build ISO ********* -Now as you are aware of the prerequisites we can continue and build our own -ISO from source. For this we have to fetch the latest source code from GitHub. +Now that you understand the prerequisites, you can build a VyOS ISO from source. +First, fetch the latest source code from GitHub: .. code-block:: none $ git clone -b current --single-branch https://github.com/vyos/vyos-build -Now a fresh build of the VyOS ISO can begin. Change directory to the -``vyos-build`` directory and run: +Now you can begin a fresh VyOS ISO build. Change to the ``vyos-build`` +directory and run: .. code-block:: none @@ -188,8 +179,8 @@ Start the build: vyos_bld@8153428c7e1f:/vyos$ sudo make clean vyos_bld@8153428c7e1f:/vyos$ sudo ./build-vyos-image --architecture amd64 --build-by "j.randomhacker@vyos.io" generic -When the build is successful, the resulting iso can be found inside the -``build`` directory as ``live-image-[architecture].hybrid.iso``. +When the build is successful, find the resulting ISO in the ``build`` directory +as ``live-image-[architecture].hybrid.iso``. .. _build source: @@ -199,8 +190,8 @@ When the build is successful, the resulting iso can be found inside the Customize ========= -This ISO can be customized with the following list of configure options. -The full and current list can be generated with ``./build-vyos-image --help``: +You can customize the ISO with the following configure options. Generate the +full and current list with ``./build-vyos-image --help``: .. code-block:: none @@ -411,11 +402,11 @@ file (example uses kernel 4.19.146): HEAD is now at 015e94d0e37b Linux 4.19.146 -Now we can use the helper script ``build-kernel.sh`` which does all the -necessary voodoo by applying required patches from the -`vyos-build/packages/linux-kernel/patches` folder, copying our kernel -configuration ``x86_64_vyos_defconfig`` to the right location, and finally -building the Debian packages. +Now you can use the helper script ``build-kernel.sh``, which completes all +the necessary steps: applying required patches from the +``vyos-build/packages/linux-kernel/patches`` folder, copying the kernel +configuration ``x86_64_vyos_defconfig`` to the correct location, and building +the Debian packages. .. note:: Building the kernel will take some time depending on the speed and quantity of your CPU/cores and disk speed. Expect 20 minutes @@ -496,16 +487,16 @@ building the Debian packages. dpkg-buildpackage: info: binary-only upload (no source included) -In the end you will be presented with the kernel binary packages which you can -then use in your custom ISO build process, by placing all the `*.deb` files in -the vyos-build/packages folder where they will be used automatically when -building VyOS as documented above. +When complete, you will have kernel binary packages to use in your custom ISO +build. Place all ``*.deb`` files in the ``vyos-build/packages`` folder, where +the build process will use them automatically. Firmware ^^^^^^^^ If you upgrade your kernel or include new drivers you may need new firmware. -Build a new ``vyos-linux-firmware`` package with the included helper scripts. +This builds a new ``vyos-linux-firmware`` package using the included helper +scripts. .. code-block:: none @@ -514,8 +505,8 @@ Build a new ``vyos-linux-firmware`` package with the included helper scripts. $ ./build-linux-firmware.sh $ cp vyos-linux-firmware_*.deb ../ -This tries to automatically detect which blobs are needed based on which drivers -were built. If it fails to find the correct files you can add them manually to +The script automatically detects which firmware blobs are needed based on the +built drivers. If detection fails, you can manually add files to ``vyos-build/packages/linux-kernel/build-linux-firmware.sh``: .. code-block:: bash @@ -526,24 +517,24 @@ were built. If it fails to find the correct files you can add them manually to Building Out-Of-Tree Modules ---------------------------- -Building the kernel is one part, but now you also need to build the required -out-of-tree modules so everything is lined up and the ABIs match. To do so, -you can again take a look at ``vyos-build/packages/linux-kernel/Jenkinsfile`` -to see all of the required modules and their selected versions. We will show -you how to build all the current required modules. +Building the kernel is one step. You must also build required out-of-tree +modules so the ABIs match. +Refer to ``vyos-build/packages/linux-kernel/Jenkinsfile`` +for all required modules and their versions. We show you how to build the +currently required modules. Accel-PPP ^^^^^^^^^ -First, clone the source code and check out the appropriate version by running: +First, clone the source code and check out the appropriate version: .. code-block:: none $ cd vyos-build/packages/linux-kernel $ git clone https://github.com/accel-ppp/accel-ppp.git -We again make use of a helper script and some patches to make the build work. -Just run the following command: +Use the helper script and patches to build the package. Run the following +command: .. code-block:: none @@ -576,10 +567,9 @@ to the ``vyos-build/packages`` folder for inclusion during the ISO build. Intel NIC ^^^^^^^^^ -The Intel NIC drivers do not come from a Git repository, instead we just fetch -the tarballs from our mirror and compile them. - -Simply use our wrapper script to build all of the driver modules. +The Intel NIC drivers do not come from a Git repository. VyOS fetches the +tarballs from a mirror and compiles them. Use the following wrapper script +to build all driver modules: .. code-block:: none @@ -598,17 +588,17 @@ Simply use our wrapper script to build all of the driver modules. Created package {:path=>"vyos-intel-iavf_4.0.1-0_amd64.deb"} I: Cleanup iavf source -After compiling the packages you will find yourself the newly generated `*.deb` -binaries in ``vyos-build/packages/linux-kernel`` from which you can copy them -to the ``vyos-build/packages`` folder for inclusion during the ISO build. +After compilation, find the generated ``*.deb`` binaries in +``vyos-build/packages/linux-kernel``. Copy them to the ``vyos-build/packages`` +folder for inclusion in the ISO build. Intel QAT ^^^^^^^^^ -The Intel QAT (Quick Assist Technology) drivers do not come from a Git -repository, instead we just fetch the tarballs from 01.org, Intel's -open-source website. -Simply use our wrapper script to build all of the driver modules. +The Intel QAT (Quick Assist Technology) drivers do not come from a Git +repository. VyOS fetches the tarballs from ``01.org``, Intel's open-source +website. +Use the following wrapper script to build all driver modules: .. code-block:: none @@ -640,24 +630,23 @@ to the ``vyos-build/packages`` folder for inclusion during the ISO build. Packages ======== -If you are brave enough to build yourself an ISO image containing any modified -package from our GitHub organisation - this is the place to be. +If you are brave enough to build your own ISO image with any modified package +from VyOS's GitHub organisation, this is the place for you. -Any "modified" package may refer to an altered version of e.g. vyos-1x package -that you would like to test before filing a pull request on GitHub. +Any modified package may be an altered version (e.g., ``vyos-1x``) that you +want to test before filing a pull request on GitHub. -Building an ISO with any customized package is in no way different than -building a regular (customized or not) ISO image. Simply place your modified -`*.deb` package inside the `packages` folder within `vyos-build`. The build -process will then pickup your custom package and integrate it into your ISO. +Building an ISO with a customized package is the same as building a regular +ISO image. Place your modified ``*.deb`` package inside the ``packages`` folder +within ``vyos-build``. The build process will automatically use your custom +package during the ISO build. Troubleshooting =============== -Debian APT is not very verbose when it comes to errors. If your ISO build breaks -for whatever reason and you suspect it's a problem with APT dependencies or -installation you can add this small patch which increases the APT verbosity -during ISO build. +Debian APT does not provide verbose error messages. If your ISO build fails and +you suspect an APT dependencies or installation issue, you can apply this patch +to increase APT verbosity during the ISO build. .. stop_vyoslinter @@ -686,17 +675,17 @@ during ISO build. Packages ******** -VyOS itself comes with a bunch of packages that are specific to our system and -thus cannot be found in any Debian mirror. Those packages can be found at the -`VyOS GitHub project`_ in their source format can easily be compiled into -a custom Debian (`*.deb`) package. +VyOS comes with specific packages that cannot be found in any +Debian mirror. These packages are located in the `VyOS GitHub project`_ in +source format and can easily be compiled into custom +Debian (``*.deb``) packages. -The easiest way to compile your package is with the above mentioned -:ref:`build_docker` container, it includes all required dependencies for -all VyOS related packages. +The easiest way to compile your package is with the :ref:`build_docker` +container mentioned earlier, as it includes all required dependencies for all +VyOS related packages. -Assume we want to build the vyos-1x package on our own and modify it to our -needs. We first need to clone the repository from GitHub. +Assuming you want to build the ``vyos-1x`` package and modify it for your needs, +first clone the repository from GitHub: .. code-block:: none @@ -705,7 +694,7 @@ needs. We first need to clone the repository from GitHub. Build ===== -Launch Docker container and build package +Launch the Docker container and build the package: .. code-block:: none @@ -718,8 +707,8 @@ Launch Docker container and build package # Build DEB $ dpkg-buildpackage -uc -us -tc -b -After a minute or two you will find the generated DEB packages next to the -vyos-1x source directory: +After a minute or two, the generated DEB packages are located next to the +``vyos-1x`` source directory: .. code-block:: none @@ -730,11 +719,10 @@ vyos-1x source directory: Install ======= -To take your newly created package on a test drive you can simply SCP it to a -running VyOS instance and install the new `*.deb` package over the current -running one. +To test your newly created package, you can SCP it to a running VyOS instance +and install the new ``*.deb`` package to replace the current one. -Just install using the following commands: +Install the package using the following commands: .. code-block:: none @@ -745,14 +733,14 @@ Just install using the following commands: Setting up vyos-1x (1.3dev0-1847-gb6dcb0a8) ... Processing triggers for rsyslog (8.1901.0-1) ... -You can also place the generated `*.deb` into your ISO build environment to -include it in a custom iso, see :ref:`build_custom_packages` for more +You can also place the generated ``*.deb`` in your ISO build environment to +include it in a custom ISO. See :ref:`build_custom_packages` for more information. -.. warning:: Any packages in the packages directory will be added to the iso - during build, replacing the upstream ones. Make sure you delete them (both - the source directories and built deb packages) if you want to build an iso - from purely upstream packages. +.. warning:: Any packages in the ``packages`` directory will be added to the + ISO during the build, replacing upstream packages. Delete both the source + directories and built DEB packages if you want to build an ISO from purely + upstream packages. .. stop_vyoslinter diff --git a/docs/contributing/testing.rst b/docs/contributing/testing.rst index ea9a683b41..d6a68ee33a 100644 --- a/docs/contributing/testing.rst +++ b/docs/contributing/testing.rst @@ -28,7 +28,7 @@ Smoketests ========== Smoketests execute predefined VyOS CLI commands and check if the desired -daemon/service configuration is rendert - that is how to put it "short". +daemon or service configuration is rendered. When an ISO image is assembled by the `VyOS CI`_, the ``BUILD_SMOKETEST`` parameter is enabled by default. This extends the ISO configuration line @@ -49,7 +49,7 @@ QEMU instance, and the ISO image is first installed to the virtual hard disk. After the first boot into the newly installed system, the main Smoketest script is executed. It can be found at `/usr/bin/vyos-smoketest`. -The script only searches for executable "test-cases" under +The script searches for executable test cases under ``/usr/libexec/vyos/tests/smoke/cli/`` and executes them one by one. .. note:: Smoketests will alter the system configuration. If you are logged @@ -88,7 +88,7 @@ Example: OK -Interface based tests +Interface-based tests --------------------- Our smoketests not only test daemons and services, but also check if interface From 631a97657f0b54def316bbf0d2942d8314cc7a31 Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Fri, 5 Dec 2025 21:17:34 +0900 Subject: [PATCH 05/10] Fix line length errors --- docs/contributing/build-vyos.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/docs/contributing/build-vyos.rst b/docs/contributing/build-vyos.rst index 003104b272..65298a7a78 100644 --- a/docs/contributing/build-vyos.rst +++ b/docs/contributing/build-vyos.rst @@ -10,16 +10,19 @@ Build VyOS Prerequisites ************* -There are different ways you can build VyOS. Building using a :ref:`build_docker` +There are different ways you can build VyOS. Building using a +:ref:`build_docker` container is the easiest way because all dependencies are managed for you. -Alternatively, you can set up your own build machine and run a :ref:`build_native` build. +Alternatively, you can set up your own build machine and run a +:ref:`build_native` build. -.. note:: Starting with VyOS 1.4, only source code and Debian package repositories - of the rolling release (the **current** branch) are publicly available. +.. note:: Starting with VyOS 1.4, only source code and Debian package + repositories of the rolling release (the **current** branch) are publicly + available. The source code and pre-built Debian package repositories of LTS releases - are only available to subscription holders (customers and active community members - with contributors subscriptions). + are only available to subscription holders (customers and active community + members with contributors subscriptions). The following includes the build process for VyOS rolling release. @@ -41,8 +44,9 @@ To get started, clone the repository to your local machine: $ sudo make clean $ sudo ./build-vyos-image --architecture amd64 --build-by "j.randomhacker@vyos.io" generic -For required packages, refer to the ``docker/Dockerfile`` file in the repository_. -The ``./build-vyos-image`` script will also warn you if any dependencies are missing. +For required packages, refer to the ``docker/Dockerfile`` file in the +repository_. The ``./build-vyos-image`` script will also warn you if any +dependencies are missing. .. _build_docker: From 2a176817e12a12833c07bd080d5816fe83725ca7 Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Sat, 6 Dec 2025 13:44:02 +0900 Subject: [PATCH 06/10] Proofread debugging.rst --- docs/contributing/cla.rst | 31 ++--- docs/contributing/debugging.rst | 193 ++++++++++++++++---------------- 2 files changed, 110 insertions(+), 114 deletions(-) diff --git a/docs/contributing/cla.rst b/docs/contributing/cla.rst index 778beb9d1a..4396a426ba 100644 --- a/docs/contributing/cla.rst +++ b/docs/contributing/cla.rst @@ -1,3 +1,5 @@ +:lastproofread: 2025-12-05 + .. _cla: ############################# @@ -7,38 +9,37 @@ Contributor License Agreement Before we can accept your contributions to VyOS, you must sign a **Contributor License Agreement (CLA)**. -This is a standard open-source practice designed to protect both you and the -project. +This is a standard open-source practice that protects both you and the project. -The process is simple and fully automated: +The process is straightforward and fully automated: 1. **Review the CLA document** - You can find the CLA text at our + Find the CLA text in our `GitHub repository `__. 2. **Submit a pull request** - When you open a pull request, a CLA bot will automatically check whether all + When you open a pull request, a CLA bot automatically checks whether all commit authors have signed the CLA. 3. **Follow the bot's instructions** - If the CLA has not been signed, the bot will leave a comment with a prompt. - Simply reply to that comment with the suggested text to sign the CLA. + If the CLA has not been signed, the bot leaves a comment with instructions. + Reply to that comment with the suggested text to sign the CLA. 4. **Wait for confirmation** - The CLA bot will verify your response and update the pull request status. - Once all commit authors have signed, the bot will confirm that the CLA - requirement has been met, and unlock the pull request for merging. + The CLA bot verifies your response and updates the pull request status. + Once all commit authors have signed, the bot confirms that the CLA + requirement is met and unlocks the pull request for merging. .. note:: - The CLA must be signed by **each commit author**. + Each commit author must sign the CLA. - If your pull request includes commits from multiple contributors, each of - them must sign the CLA before the pull request can be accepted. + If your pull request includes commits from multiple contributors, each one + must sign the CLA before the pull request can be accepted. -Once signed, the CLA remains valid for all past and future contributions to VyOS -made under the same GitHub identity. +Once you sign the CLA, it remains valid for all your past and future contributions +to VyOS under the same GitHub identity. diff --git a/docs/contributing/debugging.rst b/docs/contributing/debugging.rst index e443feee73..41aa3c32fd 100644 --- a/docs/contributing/debugging.rst +++ b/docs/contributing/debugging.rst @@ -1,116 +1,113 @@ +:lastproofread: 2025-12-05 + .. _debugging: ######### Debugging ######### -There are two flags available to aid in debugging configuration scripts. -Since configuration loading issues will manifest during boot, the flags are -passed as kernel boot parameters. +Two flags are available to help debug configuration scripts. Configuration +loading issues manifest during boot, so these flags are passed as kernel boot +parameters. ISO image build =============== -When having trouble compiling your own ISO image or debugging Jenkins issues -you can follow the steps at :ref:`iso_build_issues`. +If you have trouble compiling your own ISO image or debugging Jenkins issues, +follow the steps at :ref:`iso_build_issues`. System Startup ============== -The system startup can be debugged (like loading in the configuration -file from ``/config/config.boot``. This can be achieve by extending the -Kernel command-line in the bootloader. +Debug system startup by examining the configuration file loading from +``/config/config.boot``. Extend the kernel command-line in the bootloader to +enable this. Kernel ------ -* ``vyos-debug`` - Adding the parameter to the linux boot line will produce - timing results for the execution of scripts during commit. If one is seeing - an unexpected delay during manual or boot commit, this may be useful in - identifying bottlenecks. The internal flag is ``VYOS_DEBUG``, and is found - in vyatta-cfg_. Output is directed to ``/var/log/vyatta/cfg-stdout.log``. +* ``vyos-debug`` - Add this parameter to the Linux boot line to produce + timing results for script execution during commit. If you see an unexpected + delay during manual or boot commit, this parameter helps identify bottlenecks. + The internal flag is ``VYOS_DEBUG``, found in vyatta-cfg_. Output is directed + to ``/var/log/vyatta/cfg-stdout.log``. -* ``vyos-config-debug`` - During development, coding errors can lead to a - commit failure on boot, possibly resulting in a failed initialization of the - CLI. In this circumstance, the kernel boot parameter ``vyos-config-debug`` - will ensure access to the system as user ``vyos``, and will log a Python - stack trace to the file ``/tmp/boot-config-trace``. - File ``boot-config-trace`` will generate only if config loaded with a failure - status. +* ``vyos-config-debug`` - During development, coding errors can cause commit + failures on boot, potentially preventing CLI initialization. This kernel boot + parameter ensures access to the system as user ``vyos`` and logs a Python + stack trace to ``/tmp/boot-config-trace``. The file is created only if the + configuration load fails. Live System =========== -A number of flags can be set up to change the behaviour of VyOS at runtime. -These flags can be toggled using either environment variables or creating -files. +Several flags can be set to change VyOS behavior at runtime. Toggle these flags +using environment variables or by creating files. -For each feature, a file called ``vyos.feature.debug`` can be created to -toggle the feature on. If a parameter is required it can be placed inside -the file as its first line. +For each feature, create a file called ``vyos.feature.debug`` to enable it. +If a parameter is required, place it as the first line inside the file. -The file can be placed in ``/tmp`` for one time debugging (as the file -will be removed on reboot) or placed in '/config' to stay permanently. +Place the file in ``/tmp`` for one-time debugging (the file is removed on +reboot) or in ``/config`` to persist permanently. For example, ``/tmp/vyos.ifconfig.debug`` can be created to enable interface debugging. -It is also possible to set up the debugging using environment variables. -In that case, the name will be (in uppercase) VYOS_FEATURE_DEBUG. +You can also enable debugging using environment variables. +The environment variable name follows the convention ``VYOS_FEATURE_DEBUG``. -For example running, ``export VYOS_IFCONFIG_DEBUG=""`` on your vbash, -will have the same effect as ``touch /tmp/vyos.ifconfig.debug``. +For example, ``export VYOS_IFCONFIG_DEBUG=""`` in your vbash has the same effect +as ``touch /tmp/vyos.ifconfig.debug``. -* ``ifconfig`` - Once set, all commands used, and their responses received - from the OS, will be presented on the screen for inspection. +* ``ifconfig`` - Display all commands and their responses from the OS on + screen for inspection. -* ``command`` - Once set, all commands used, and their responses received - from the OS, will be presented on the screen for inspection. +* ``command`` - Display all commands and their responses from the OS on screen + for inspection. -* ``developer`` - Should a command fail, instead of printing a message to the - user explaining how to report issues, the python interpreter will start a - PBD post-mortem session to allow the developer to debug the issue. As the - debugger will wait from input from the developer, it has the capacity to - prevent a router to boot and therefore should only be permanently set up - on production if you are ready to see the OS fail to boot. +* ``developer`` - When a command fails, start a PDB post-mortem session instead + of showing a standard error message. This allows developers to debug issues + interactively. Because the debugger waits for input, it can prevent the router + from booting, so only enable this permanently on production systems if you are + ready for potential boot failures. -* ``log`` - In some rare cases, it may be useful to see what the OS is doing, - including during boot. This option sends all commands used by VyOS to a - file. The default file is ``/tmp/full-log`` but it can be changed. +* ``log`` - Send all commands used by VyOS to a log file for inspection. This + is useful in rare cases when you need to see what the OS is doing, including + during boot. The default file is ``/tmp/full-log``, but you can change it. -.. note:: In order to retrieve the debug output on the command-line you need to - disable ``vyos-configd`` in addition. This can be run either one-time by - calling ``sudo systemctl stop vyos-configd`` or make this reboot-safe by - calling ``sudo systemctl disable vyos-configd``. +.. note:: To retrieve debug output on the command line, disable ``vyos-configd`` + in addition. You can do this one-time with ``sudo systemctl stop vyos-configd`` + or permanently with ``sudo systemctl disable vyos-configd``. FRR --- -Recent versions use the ``vyos.frr`` framework. The Python class is located -inside our ``vyos-1x:python/vyos/frr.py``. It comes with an embedded debugging/ -(print style) debugger as vyos.ifconfig does. +Recent versions use the ``vyos.frr`` framework. The Python class is located in +``vyos-1x:python/vyos/frr.py``. It includes an embedded debugger similar to the +one in ``vyos.ifconfig``. -To enable debugging just run: ``$ touch /tmp/vyos.frr.debug`` +Enable debugging by running: ``touch /tmp/vyos.frr.debug`` -Debugging Python Code with PDB +Debug Python code with PDB ------------------------------ -Sometimes it might be useful to debug Python code interactively on the live -system rather than a IDE. This can be achieved using pdb. +Sometimes it is useful to debug Python code interactively on the live system +rather than in an IDE. You can do this using pdb. -Let us assume you want to debug a Python script that is called by an op-mode -command. After you found the script by looking up the op-mode-defitions you -can edit the script in the live system using e.g. vi: +Assuming you want to debug a Python script called by an op-mode command, find +the script by looking up the op-mode definitions, then edit it on the live +system using vi: ``vi /usr/libexec/vyos/op_mode/show_xyz.py`` Insert the following statement right before the section where you want to -investigate a problem (e.g. a statement you see in a backtrace): +investigate a problem (for example, a statement you see in a backtrace): ``import pdb; pdb.set_trace()`` -Optionally you can surrounded this statement by an ``if`` which only triggers -under the condition you are interested in. -Once you run ``show xyz`` and your condition is triggered you should be dropped -into the python debugger: +Optionally, surround this statement with an ``if`` condition that triggers only +for the conditions you are interested in. + +When you run ``show xyz`` and your condition triggers, you enter the Python +debugger: .. code-block:: none @@ -122,7 +119,7 @@ into the python debugger: You can type ``help`` to get an overview of the available commands, and ``help command`` to get more information on each command. -Useful commands are: +Common useful commands include: * examine variables using ``pp(var)`` * continue execution using ``cont`` @@ -131,9 +128,9 @@ Useful commands are: Config Migration Scripts ------------------------ -Starting with VyOS 1.5 a new mechanism is used for config migration whichwill improve -migration performance. New migrators only exist in the new format with a migration() -function. +Starting with VyOS 1.5, a new mechanism is used for config migration that +improves migration performance. New migrators use only the new format with a +``migration()`` function. .. code-block:: python @@ -145,9 +142,8 @@ function. return # do your stuff here -New style migrations scripts can no longer be executed on their own. The new -handler of the entire migration subsystem on the other hand comes with a handy -test kit: +New-style migration scripts can no longer run on their own. However, the new +migration subsystem handler includes a test kit: .. code-block:: none @@ -166,57 +162,56 @@ test kit: --force force run of all migration scripts -So in order to test your migrator you can run this as simple as: +To test your migration, run: .. code-block:: none vyos@vyos:~$ /usr/libexec/vyos/run-config-migration.py --test-script /opt/vyatta/etc/config-migrate/migrate/quagga/11-to-12 --output-file /tmp/foo /tmp/static-route-basic vyos@vyos:~$ cat /tmp/foo -Where `/tmp/foo` will contain the migrated configuration. +The file ``/tmp/foo`` contains the migrated configuration. Configuration Error on System Boot ---------------------------------- -Being brave and running the latest rolling releases will sometimes trigger -bugs due to corner cases we missed in our design. Those bugs should be filed -via Phabricator_ but you can help us to narrow down the issue. Login to your -VyOS system and change into configuration mode by typing ``configure``. Now -re-load your boot configuration by simply typing ``load`` followed by return. +Running the latest rolling releases sometimes exposes bugs due to edge cases +missed in design. File these bugs via Phabricator_, but you can help narrow +down the issue by following these steps: + +1. Log in to your VyOS system. +2. Enter configuration mode: ``configure`` +3. Reload your boot configuration: ``load`` -You should now see a Python backtrace which will help us to handle the issue, -please attach it to the Phabricator_ task. +You should see a Python backtrace that helps identify the issue. Attach it to +the Phabricator_ task. Boot Timing ----------- -During the migration and extensive rewrite of functionality from Perl into -Python a significant increase in the overall system boottime was noticed. The -system boot time can be analysed and a graph can be generated in the end which -shows in detail who called whom during the system startup phase. +During the migration and rewrite of functionality from Perl to Python, system +boot time increased significantly. You can analyze and graph boot time to see +detailed call sequences during startup. -This is done by utilizing the ``systemd-bootchart`` package which is now -installed by default on the VyOS 1.3 (equuleus) branch. The configuration is -also versioned so we get comparable results. ``systemd-bootchart`` is configured -using this file: bootchart.conf_ +This uses the ``systemd-bootchart`` package, which is installed by default on +VyOS 1.3 (equuleus) and later. Configuration is versioned for comparable results. +Refer to bootchart.conf_ for the configuration file. -To enable boot time graphing change the Kernel commandline and add the following -string: ``init=/usr/lib/systemd/systemd-bootchart`` +To enable boot time graphing, add the following to the kernel command line: +``init=/usr/lib/systemd/systemd-bootchart`` -This can also be done permanently by changing ``/boot/grub/grub.cfg``. +You can also make this permanent by editing ``/boot/grub/grub.cfg``. Priorities ========== -VyOS CLI is all about priorities. Every CLI node has a corresponding -``node.def`` file and possibly an attached script that is executed when the -node is present. Nodes can have a priority, and on system bootup - or any -other ``commit`` to the config all scripts are executed from lowest to highest -priority. This is good as this gives a deterministic behavior. +VyOS CLI depends heavily on priorities. Every CLI node has a corresponding +``node.def`` file and possibly an attached script. Nodes can have priorities, +and on system bootup or any ``commit`` to the configuration, scripts execute +from lowest to highest priority. This provides deterministic behavior. -To debug issues in priorities or to see what's going on in the background -you can use the ``/opt/vyatta/sbin/priority.pl`` script which lists to you -the execution order of the scripts. +To debug priority issues or see script execution order, use the +``/opt/vyatta/sbin/priority.pl`` script, which lists the execution order of +scripts. .. stop_vyoslinter From 76a3a48dc345c819de5b31542c66badd2a4156fd Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Mon, 8 Dec 2025 14:27:32 +0900 Subject: [PATCH 07/10] Proofread issues-features.rst --- docs/contributing/cla.rst | 4 +- docs/contributing/issues-features.rst | 138 ++++++++++++-------------- 2 files changed, 66 insertions(+), 76 deletions(-) diff --git a/docs/contributing/cla.rst b/docs/contributing/cla.rst index 4396a426ba..c19e266a26 100644 --- a/docs/contributing/cla.rst +++ b/docs/contributing/cla.rst @@ -41,5 +41,5 @@ The process is straightforward and fully automated: If your pull request includes commits from multiple contributors, each one must sign the CLA before the pull request can be accepted. -Once you sign the CLA, it remains valid for all your past and future contributions -to VyOS under the same GitHub identity. +Once you sign the CLA, it remains valid for all your past and future +contributions to VyOS under the same GitHub identity. diff --git a/docs/contributing/issues-features.rst b/docs/contributing/issues-features.rst index e57766fc83..bc62388144 100644 --- a/docs/contributing/issues-features.rst +++ b/docs/contributing/issues-features.rst @@ -1,3 +1,5 @@ +:lastproofread: 2025-12-08 + .. _issues_features: ####################### @@ -9,17 +11,15 @@ Issues/Feature requests Bug Report/Issue ================ -Issues or bugs are found in any software project. VyOS is not an exception. +Issues and bugs occur in every software project, and VyOS is no exception. -All issues should be reported to the developers. This lets the developers know -what is not working properly. Without this sort of feedback every developer -will believe that everything is working correctly. +Report all issues to the developers so they know what is not working correctly. +Without this feedback, developers assume everything works properly. I have found a bug, what should I do? ------------------------------------- -When you believe you have found a bug, it is always a good idea to verify the -issue prior to opening a bug request. +When you believe you have found a bug, verify it first to ensure it is genuine. * Consult the documentation_ to ensure that you have configured your system correctly @@ -28,90 +28,80 @@ issue prior to opening a bug request. Ensure the problem is reproducible ---------------------------------- -You should include the following information: - -* A sequence of configuration commands or a complete configuration file - required to recreate a setup where the bug occurs. - Please avoid partial configs: a sequence of commands is easy to paste into the console, - a complete config is easy to load in a VM, but a partial config is neither! - At least not until we implement a "merge from the CLI" - feature that allows pasting config file chunks into a session. -* The behavior you expect and how it's different from the behavior you observe. - Don't just include command outputs or traffic dumps — - try to explain at least briefly why they are wrong and what they should be. -* A sequence of actions that triggers the bug. - We understand that it's not always possible, but it makes developer's job a lot easier - and also allows any community member to independently confirm - that the bug still exists or if it's already fixed. -* If it's a regression, tell us a VyOS version where the feature still worked correctly. - It's perfect if you can tell exactly which version broke it, - but we understand that it's not always easy or feasible — any working version is acceptable. - -If you aren't certain what the correct behavior is and if what you see is really a bug, -or if you don't have a reproducing procedure that reliably triggers it, -please create a post on the forum or ask in the chat first — -or, if you have a subscription, create a support ticket. -Our team and community members can help you identify the bug and work around it, -then create an actionable and testable bug report. +Include the following information when reporting a bug: + +* A sequence of configuration commands or a complete configuration file needed + to recreate the bug. Avoid partial configs: a sequence of commands is easy to + paste, a complete config is easy to load, but a partial config is neither. + This limitation should improve once we implement a "merge from the CLI" feature. +* Describe the expected behavior and how it differs from what you observe. Include + command outputs or traffic dumps, but explain briefly why they are wrong and + what the correct behavior should be. +* A sequence of actions that trigger the bug. While not always possible, this + helps developers and community members confirm the issue and verify fixes. +* If it is a regression, specify a VyOS version where the feature worked correctly. + If you can identify the exact version that broke it, that is helpful. Any + working version is acceptable. + +If you are uncertain whether the behavior is a bug or what the correct behavior +is, or if you lack a reliable reproducing procedure, post on the forum or ask in +chat first. If you have a subscription, create a support ticket. The team and +community can help identify the issue, work around it, and create an actionable +bug report. Report a Bug ------------ -In order to open up a bug-report/feature request you need to create yourself -an account on VyOS Phabricator_. On the left side of the specific project (VyOS -1.2 or VyOS 1.3) you will find quick-links for opening a bug-report/feature -request. +To open a bug report or feature request, create an account on VyOS Phabricator_. +On the left side of the specific project (VyOS 1.2 or VyOS 1.3), you will find +quick links for opening bug reports or feature requests. -* Provide as much information as you can -* Which version of VyOS are you using? ``run show version`` -* How can we reproduce this Bug? +* Provide as much information as you can. +* Specify which VyOS version you are using: ``run show version`` +* Explain how to reproduce the bug. .. _feature_request: Feature Requests ================ -You have an idea of how to make VyOS better or you are in need of a specific -feature which all users of VyOS would benefit from? To send a feature request -please search Phabricator_ to check if there is already a request pending. You can -enhance it or if you don't find one, create a new one by use the quick link in -the left side under the specific project. +Have an idea to improve VyOS or need a feature that would benefit all users? +Before submitting a feature request, search Phabricator_ to check if a request +already exists. You can enhance an existing request or create a new one using +the quick link on the left side of the specific project. -You must create a task before you start working on a feature. -Yes, even if it's a tiny feature — we use the task tracker to generate release notes, -so it's essential that everything is reflected there. +Create a task before starting work on a feature, even if it is a tiny feature. +We use the task tracker to generate release notes, so all work must be reflected +there. -You must include at least the following: +Include at least the following information: -* A reasonably detailed description of the feature: what it is, how it's supposed to work, - and how you'd use it. - The maintainers aren't familiar with every feature of every protocol and tool, - and community contributors who are looking for tasks to work on will also - appreciate more information that helps them implement and test a feature. -* Proposed CLI syntax, if the feature requires new commands. - Please include both configuration and operational mode commands, if both are required. +* Provide a detailed description of the feature: what it is, how it works, and how + you would use it. Maintainers may not be familiar with every feature of every + protocol and tool. Community contributors looking for work also appreciate + detailed information that helps them implement and test the feature. +* Include proposed CLI syntax if the feature requires new commands. Provide both + configuration and operational mode commands if both are needed. -You should include the following information: +Consider including the following information: -* Is the feature supported by the underlying component - (FreeRangeRouting, nftables, Kea...) already? -* How you'd configure it by hand there? -* Are there any limitations (hardware support, resource usage)? -* Are there any adverse or non-obvious interactions with other features? - Should it be mutually exclusive with anything? +* Is the feature already supported by the underlying component (FreeRangeRouting, + nftables, Kea, etc.)? +* How would you configure it manually with that component? +* Are there limitations (hardware support, resource usage)? +* Are there any adverse or non-obvious interactions with other features? Should + it be mutually exclusive with anything? -It's fine if you cannot provide some of that information, but if you can, -it makes the work of developers considerably simpler, -so try to do the research to answer those questions. +You do not need to provide all this information, but if you can, it simplifies +developers' work considerably. Research these questions when possible. Task auto-closing ================= -There is a special status for tasks -where all work on the side of maintainers and contributors is complete: -"Needs reporter action". +A special task status exists for when all work by maintainers and contributors +is complete: "Needs reporter action". -We assign that status to: +We assign this status to: * Feature requests that do not include required information and need clarification. * Bug reports that lack reproducing procedures. @@ -119,14 +109,14 @@ We assign that status to: but require testing in the real-world environment that only the reporter can replicate (e.g., hardware we do not have, specific network conditions...). -This is what will happen when a task is set to "Needs reporter action": +When a task is set to "Needs reporter action", the following happens: -* If there is no response from the reporter within two weeks, - the task bot will add a comment ("Any news?") to remind the reporter to reply. -* If there is no response after further two weeks, the task will be automatically closed. +* If no response arrives within two weeks, the task bot adds a comment + ("Any news?") to remind the reporter. +* If there is still no response after two more weeks, the task closes automatically. -We will not auto-close tasks with any other status -and will not close tasks for the lack of maintainer activity! +We do not auto-close tasks with any other status and do not close tasks due to +lack of maintainer activity. .. _documentation: https://docs.vyos.io .. _Slack: https://slack.vyos.io From afb70add93da82805d2823f9c4158ac1e27b3eb9 Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Thu, 11 Dec 2025 21:53:07 +0900 Subject: [PATCH 08/10] Complete issuesfeatures.rst proofreading --- docs/contributing/issues-features.rst | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/docs/contributing/issues-features.rst b/docs/contributing/issues-features.rst index bc62388144..eabfbe6178 100644 --- a/docs/contributing/issues-features.rst +++ b/docs/contributing/issues-features.rst @@ -14,7 +14,7 @@ Bug Report/Issue Issues and bugs occur in every software project, and VyOS is no exception. Report all issues to the developers so they know what is not working correctly. -Without this feedback, developers assume everything works properly. +Without this feedback, developers may assume a broken feature works properly. I have found a bug, what should I do? ------------------------------------- @@ -53,8 +53,8 @@ Report a Bug ------------ To open a bug report or feature request, create an account on VyOS Phabricator_. -On the left side of the specific project (VyOS 1.2 or VyOS 1.3), you will find -quick links for opening bug reports or feature requests. +On the left side of the specific project (VyOS 1.2, VyOS 1.3, or VyOS 1.4), +you will find links for opening bug reports and feature requests. * Provide as much information as you can. * Specify which VyOS version you are using: ``run show version`` @@ -76,17 +76,18 @@ there. Include at least the following information: -* Provide a detailed description of the feature: what it is, how it works, and how - you would use it. Maintainers may not be familiar with every feature of every - protocol and tool. Community contributors looking for work also appreciate - detailed information that helps them implement and test the feature. +* Provide a detailed description of the feature: what it is, how it works, and + how you would use it. Maintainers may not be familiar with every feature of + every protocol and tool. Community contributors looking for work also + appreciate detailed information that helps them implement and test the + feature. * Include proposed CLI syntax if the feature requires new commands. Provide both configuration and operational mode commands if both are needed. Consider including the following information: -* Is the feature already supported by the underlying component (FreeRangeRouting, - nftables, Kea, etc.)? +* Is the feature already supported by the underlying component + (FreeRangeRouting, nftables, Kea, etc.)? * How would you configure it manually with that component? * Are there limitations (hardware support, resource usage)? * Are there any adverse or non-obvious interactions with other features? Should From c5418dd57b38d3a8a56234778c63b107f1a6cff5 Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Thu, 11 Dec 2025 23:18:42 +0900 Subject: [PATCH 09/10] Initial proofread development.rst --- docs/contributing/development.rst | 624 ++++++++++++++---------------- 1 file changed, 291 insertions(+), 333 deletions(-) diff --git a/docs/contributing/development.rst b/docs/contributing/development.rst index 22bebd9c95..b4993b6802 100644 --- a/docs/contributing/development.rst +++ b/docs/contributing/development.rst @@ -4,224 +4,196 @@ Development ########### -All VyOS source code is hosted on GitHub under the VyOS organization which can -be found here: https://github.com/vyos - -Our code is split into several modules. VyOS is composed of multiple individual -packages, some of them are forks of upstream packages and are periodically -synced with upstream, so keeping the whole source under a single repository -would be very inconvenient and slow. There is now an ongoing effort to -consolidate all VyOS-specific framework/config packages into vyos-1x package, -but the basic structure is going to stay the same, just with fewer and fewer -packages while the base code is rewritten from Perl/BASH into Python using and -XML based interface definition for the CLI. - -The repository that contains all the ISO build scripts is: -https://github.com/vyos/vyos-build - -The README.md file will guide you to use the this top level repository. - -Submit a Patch -============== +Learn how to contribute to VyOS. + +.. _architecture_overview: + +Architecture Overview +===================== +All VyOS source code is hosted on GitHub in the VyOS organization: +https://github.com/vyos + +VyOS is composed of multiple modules spread across different +repositories. Some modules contain forks of upstream +packages and are periodically synced. +VyOS consolidates most packages into the +`vyos-1x `__ +repository while maintaining a consistent structure. +The base code is being rewritten +from Perl and Bash to Python using an XML-based CLI interface definition. + +VyOS ISO build scripts are hosted in the +`vyos-build `__ repository. See the +``vyos-build`` repository +`README.md file `__ +for more information on building VyOS ISO images. + +Contributing Code +================= .. warning:: - Please read and sign the :doc:`Contributor License Agreement` before - submitting any patches. - -Patches are always more than welcome. To have a clean and easy to maintain -repository we have some guidelines when working with Git. A clean repository -eases the automatic generation of a changelog file. + You must sign the :doc:`Contributor License Agreement` + for your contributions to be accepted. -A good approach for writing commit messages is actually to have a look at the -file(s) history by invoking ``git log path/to/file.txt``. +VyOS is open-source and welcomes patches. +All submissions must adhere to these guidelines: -.. _prepare_commit: - -Prepare patch/commit --------------------- +* Each commit addresses a single issue or feature. +* Each commit message references a Phabricator_ task ID + (for example, ``T1234``). +* Each commit includes a username and email to identify the author. +* Only submit bugfixes in packages other than https://github.com/vyos/vyos-1x. +* Commits follow the `coding guidelines`_ outlined below. -In a big system, such as VyOS, that is comprised of multiple components, it's -impossible to keep track of all the changes and bugs/feature requests in one's -head. We use a bugtracker known as Phabricator_ for it ("issue tracker" would -be a better term, but this one stuck). -The information is used in three ways: +Determining Package Ownership +----------------------------- -* Keep track of the progress (what we've already done in this branch and what - we still need to do). +To determine which VyOS package contains a file you want to modify, use Debian's +``dpkg -S`` command on your running VyOS installation. -* Prepare release notes for upcoming releases - -* Help future maintainers of VyOS (it could be you!) to find out why certain - things have been changed in the codebase or why certain features have been - added - -To make this approach work, every change must be associated with a task number -(prefixed with **T**) and a component. If there is no bug report/feature request -for the changes you are going to make, you have to create a Phabricator_ task -first. Once there is an entry in Phabricator_, you should reference its id in -your commit message, as shown below: - -* ``ddclient: T1030: auto create runtime directories`` -* ``Jenkins: add current Git commit ID to build description`` - -If there is no Phabricator_ reference in the commits of your pull request, we -have to ask you to amend the commit message. Otherwise we will have to reject -it. +Submitting Your Code +-------------------- -Writing good commit messages -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Fork the repository and submit a GitHub pull request. This is the preferred way +to contribute changes to VyOS. -The format should be and is inspired by: https://git-scm.com/book/ch5-2.html -It is also worth reading https://chris.beams.io/posts/git-commit/ +To fork a VyOS repository: -* A single, short, summary of the commit (recommended 50 characters or less, - not exceeding 80 characters) containing a prefix of the changed component - and the corresponding Phabricator_ reference e.g. ``snmp: T1111:`` or - ``ethernet: T2222:`` - multiple components could be concatenated as in - ``snmp: ethernet: T3333`` +1. Append ``/fork`` to the repository URL on GitHub. For example, to fork + ``vyos-1x``, use: https://github.com/vyos/vyos-1x/fork -* In some contexts, the first line is treated as the subject of an email and - the rest of the text as the body. The blank line separating the summary from - the body is critical (unless you omit the body entirely); tools like rebase - can get confused if you run the two together. +2. Clone your fork or add it as a remote to your local repository: -* Followed by a message which describes all the details like: + - Clone: ``git clone https://github.com//vyos-1x.git`` + - Add remote: ``git remote add myfork https://github.com//vyos-1x.git`` - * What/why/how something has been changed, makes everyone's life easier when - working with `git bisect` +3. Configure your Git identity: - * All text of the commit message should be wrapped at 72 characters if - possible which makes reading commit logs easier with ``git log`` on a - standard terminal (which happens to be 80x25) + .. code-block:: none - * If applicable a reference to a previous commit should be made linking - those commits nicely when browsing the history: ``After commit abcd12ef - ("snmp: this is a headline") a Python import statement is missing, - throwing the following exception: ABCDEF`` + git config --global user.name "J. Random Hacker" + git config --global user.email "jrhacker@example.net" -* Always use the ``-x`` option to the ``git cherry-pick`` command when back or - forward porting an individual commit. This automatically appends the line: - ``(cherry picked from commit )`` to the original authors commit message - making it easier when bisecting problems. +4. Make your changes and add files to the Git index: -* Every change set must be consistent (self containing)! Do not fix multiple - bugs in a single commit. If you already worked on multiple fixes in the same - file use `git add --patch` to only add the parts related to the one issue - into your upcoming commit. + - Single file: ``git add myfile`` + - Directory: ``git add somedir/*`` -Limits: +5. Commit your changes with a meaningful headline and Phabricator_ reference: -* We only accept bugfixes in packages other than https://github.com/vyos/vyos-1x - as no new functionality should use the old style templates (``node.def`` and - Perl/BASH code. Use the new style XML/Python interface instead. + ``git commit`` -Please submit your patches using the well-known GitHub pull-request against our -repositories found in the VyOS GitHub organisation at https://github.com/vyos +6. Push to your fork and create a GitHub pull request: + ``git push`` -Determinine source package --------------------------- +Alternatively, you can export commits as patches and send them to +maintainers@vyos.net or attach them directly to the Phabricator_ task: -Suppose you want to make a change in a file but yet you do not know -which of the VyOS packages ship this file. You can determine the VyOS -package name in question by using Debian's ``dpkg -S`` command of your running -VyOS installation. +* Export last commit: ``git format-patch`` +* Export last two commits: ``git format-patch -2`` +Commit Messages +=============== -Fork Repository and submit Patch --------------------------------- +For guidance on writing commit messages, review the file history +with ``git log path/to/file.txt``. -Forking the repository and submitting a GitHub pull-request is the preferred -way of submitting your changes to VyOS. You can fork any VyOS repository to your -very own GitHub account by just appending ``/fork`` to any repository's URL on -GitHub. To e.g. fork the ``vyos-1x`` repository, open the following URL in your -favourite browser: https://github.com/vyos/vyos-1x/fork +Every change must be associated with a task number (prefixed with **T**) and +a component. If no bug report or feature request exists for your changes, +create a Phabricator_ task first. Reference the task ID in your commit message: -You then can proceed with cloning your fork or add a new remote to your local -repository: +* ``ddclient: T1030: auto create runtime directories`` +* ``Jenkins: add current Git commit ID to build description`` -* Clone: ``git clone https://github.com//vyos-1x.git`` +If your pull request lacks a Phabricator_ reference, maintainers will request +that you amend the commit message. -* Fork: ``git remote add myfork https://github.com//vyos-1x.git`` +Writing Good Commit Messages +----------------------------- -In order to record you as the author of the fix please identify yourself to Git -by setting up your name and email. This can be done local for this one and only -repository ``git config`` or globally using ``git config --global``. +Follow the format described in `Git documentation `__ +and `Chris Beams' guide `__. -.. code-block:: none +Commit message format: - git config --global user.name "J. Random Hacker" - git config --global user.email "jrhacker@example.net" +1. **Summary line** (50 characters recommended, 80 maximum): Include the + component + prefix and Phabricator_ reference (for example, ``snmp: T1111:`` or + ``ethernet: T2222:``). Concatenate multiple components with colons + (for example, ``snmp: ethernet: T3333``). -Make your changes and save them. Do the following for all changes files to -record them in your created Git commit: +2. **Blank line**: Separate the summary from the body. + This blank line is critical. -* Add file to Git index using ``git add myfile``, or for a whole directory: - ``git add somedir/*`` +4. **Message body** with details: -* Commit the changes by calling ``git commit``. Please use a meaningful commit - headline (read above) and don't forget to reference the Phabricator_ ID. + * Describe what changed, why, and how. This helps with ``git bisect``. + * Wrap text at 72 characters for readability with ``git log`` on an 80x25 terminal. + * Reference previous commits when applicable: ``After commit abcd12ef ("snmp: this is a headline") a Python import statement is missing, throwing the following exception: ABCDEF`` -* Submit the patch ``git push`` and create the GitHub pull-request. +5. **Cherry-pick option**: Always use the ``-x`` option when back-porting or + forward-porting commits: + ``git cherry-pick -x `` -Attach patch to Phorge task --------------------------------- + This appends ``(cherry picked from commit )`` to the commit message, + making bisecting easier. -Follow the above steps on how to "Fork repository to submit a Patch". Instead -of uploading "pushing" your changes to GitHub you can export the patches/ -commits and send it to maintainers@vyos.net or attach it directly to the bug -(preferred over email) +6. **Single responsibility**: Each commit must be self-contained. Do not fix + multiple bugs in a single commit. Use ``git add --patch`` to stage only + the parts related to one issue. -* Export last commit to patch file: ``git format-patch`` or export the last two - commits into its appropriate patch files: ``git format-patch -2`` +Constraints: +* Only bugfixes are accepted for packages other than https://github.com/vyos/vyos-1x. + New functionality must use the new XML/Python interface, not old-style templates + (``node.def`` files and Perl/Bash code). Coding Guidelines ================= -Like any other project we have some small guidelines about our source code, too. -The rules we have are not there to punish you - the rules are in place to help -us all. By having a consistent coding style it becomes very easy for new -and also longtime contributors to navigate through the sources and all the -implied logic of any one source file.. - +VyOS maintains consistent coding standards to help contributors navigate the +codebase and understand its logic. Formatting ---------- -* Python: Tabs **shall not** be used. Every indentation level should be 4 spaces -* XML: Tabs **shall not** be used. Every indentation level should be 2 spaces +* **Python**: Use 4 spaces per indentation level. Tabs **must not** be used. +* **XML**: Use 2 spaces per indentation level. Tabs **must not** be used. + +Use tools like VIM extensions (xmllint) to enforce correct indentation. Add this +to your ``.vimrc`` file: + +.. code-block:: none -.. note:: There are extensions to e.g. VIM (xmllint) which will help you to get - your indention levels correct. Add to following to your .vimrc file: - ``au FileType xml setlocal equalprg=xmllint\ --format\ --recover\ -\ - 2>/dev/null`` now you can call the linter using ``gg=G`` in command mode. + au FileType xml setlocal equalprg=xmllint\ --format\ --recover\ -\ 2>/dev/null +Then use ``gg=G`` in command mode to run the linter. -Text generation -^^^^^^^^^^^^^^^ +Text Generation +--------------- -Template processor **should** be used for generating config files. Built-in -string formatting **may** be used for simple line-oriented formats where every -line is self-contained, such as iptables rules. Template processor **must** be -used for structured, multi-line formats such as those used by ISC DHCPd. +Use a template processor for generating config files: -The default template processor for VyOS code is Jinja2_. +* **Jinja2** is the default template processor for VyOS code. +* Built-in string formatting **may** be used for simple line-oriented formats + (for example, iptables rules) where every line is self-contained. +* Template processors **must** be used for structured, multi-line formats + (for example, ISC DHCPd configuration). +Python Code +----------- -Configuration Script Structure and Behaviour --------------------------------------------- +Configuration scripts and operation mode scripts written in Python3 should +follow these guidelines: -Your configuration script or operation mode script which is also written in -Python3 should have a line break on 80 characters. This seems to be a bit odd -nowadays but as some people also work remotely or program using vi(m) this is -a fair good standard which I hope we can rely on. +* Wrap lines at 80 characters. This improves readability when browsing + GitHub on mobile devices and reads well in side-by-side diffs. -In addition this also helps when browsing the GitHub codebase on a mobile -device if you happen to be a crazy scientist. +Structure your scripts with these functions: .. code-block:: python @@ -280,109 +252,73 @@ device if you happen to be a crazy scientist. print(e) sys.exit(1) -The ``get_config()`` function must convert the VyOS config to an abstract, -internal representation. No other function is allowed to call the ``vyos.config. -Config`` object method directly. The rationale for it is that when config reads -are mixed with other logic, it's very hard to change the config syntax since -you need to weed out every occurrence of the old syntax. If syntax-specific -code is confined to a single function, the rest of the code can be left -untouched as long as the internal representation remains compatible. - -Another advantage is testability of the code. Mocking the entire config -subsystem is hard, while constructing an internal representation by hand is -way simpler. - -The ``verify()`` function takes your internal representation of the config and -checks if it's valid, otherwise it must raise ``ConfigError`` with an error -message that describes the problem and possibly suggests how to fix it. It must -not make any changes to the system. The rationale for it is again testability -and, in the future when the config backend is ready and every script is -rewritten in this fashion, ability to execute commit dry run ("commit test" -like in JunOS) and abort commit before making any changes to the system if an -error is found in any component. - -The ``generate()`` function generates config files for system components. - -The ``apply()`` function applies the generated configuration to the live -system. It should use non-disruptive reload whenever possible. It may execute -disruptive operations such as daemon process restart if a particular component -does not support non-disruptive reload, or when the expected service degradation -is minimal (for example, in case of auxiliary services such as LLDPd). In case -of high impact services such as VPN daemon and routing protocols, when non- -disruptive reload is supported for some but not all types of configuration -changes, scripts authors should make effort to determine if a configuration -change can be done in a non-disruptive way and only resort to disruptive restart -if it cannot be avoided. - -Unless absolutely necessary, configuration scripts should not modify the active -configuration of system components directly. Whenever at all possible, scripts -should generate a configuration file or files that can be applied with a single -command such as reloading a service through systemd init. Inserting statements -one by one is particularly discouraged, for example, when configuring netfilter -rules, saving them to a file and loading it with iptables-restore should always -be preferred to executing iptables directly. - -The ``apply()`` and ``generate()`` functions may ``raise ConfigError`` if, for -example, the daemon failed to start with the updated config. It shouldn't be a -substitute for proper config checking in the ``verify()`` function. All -reasonable effort should be made to verify that generated configuration is -valid and will be accepted by the daemon, including, when necessary, cross- -checks with other VyOS configuration subtrees. - -Exceptions, including ``VyOSError`` (which is raised by ``vyos.config.Config`` -on improper config operations, such as trying to use ``list_nodes()`` on a -non-tag node) should not be silenced or caught and re-raised as config error. -Sure this will not look pretty on user's screen, but it will make way better -bug reports, and help users (and most VyOS users are IT professionals) do their -own debugging as well. - -For easy orientation we suggest you take a look on the ``ntp.py`` or -``interfaces-bonding.py`` (for tag nodes) implementation. Both files can be -found in the vyos-1x_ repository. - -Other considerations: vyos-configd ----------------------------------- - -All scripts now run under the config daemon and must conform to the -following: - -1. The signature and initial four lines of ``get_config(...)`` `must` be as - above. - -2. Each of ``get_config``, ``verify``, ``apply``, ``generate`` `must` - appear, with signatures as above, even if they are a no-op. - -3. Instantiations of ``Config`` other than that in ``get_config`` `must not` - appear. - -4. The legacy function ``my_set`` `must not` appear: modifications of the - active config `should not` appear in new code (if absolutely necessary, - alternative mechanisms may be used). - -XML (used for CLI definitions) -============================== - -The bash (or better vbash) completion in VyOS is defined in *templates*. -Templates are text files (called ``node.def``) stored in a directory tree. The -directory names define the command names, and template files define the command -behaviour. Before VyOS 1.2 (crux) this files were created by hand. After a -complex redesign process_ the new style template are automatically generated -from a XML input file. - -XML interface definitions for VyOS come with a RelaxNG schema and are located -in the vyos-1x_ module. This schema is a slightly modified schema from VyConf_ -alias VyOS 2.0 So VyOS 1.2.x interface definitions will be reusable in Nextgen -VyOS Versions with very minimal changes. - -The great thing about schemas is not only that people can know the complete -grammar for certain, but also that it can be automatically verified. The -`scripts/build-command-templates` script that converts the XML definitions to -old style templates also verifies them against the schema, so a bad definition -will cause the package build to fail. I do agree that the format is verbose, but -there is no other format now that would allow this. Besides, a specialized XML -editor can alleviate the issue with verbosity. - -Example: +**``get_config()``**: This function converts VyOS config to an abstract internal +representation. No other function may call the ``vyos.config.Config`` object +directly. Limiting config reads to one function makes it easier to modify the +config syntax in the future. Additionally, this design improves testability since +you can construct an internal representation by hand rather than mocking the +entire config subsystem. + +**``verify()``**: This function validates the internal representation. It must +raise ``ConfigError`` with a descriptive message if the config is invalid. It +**must not** make any changes to the system. This design enables future features +like commit dry-run ("commit test" as in JunOS) where the system can abort a +commit before making changes. + +**``generate()``**: This function generates config files for system components. + +**``apply()``**: This function applies the generated configuration to the live +system. Prefer non-disruptive reload when possible. Disruptive operations like +daemon restarts are acceptable only when: + +* The component does not support non-disruptive reload, or +* The expected service degradation is minimal (for example, auxiliary services like LLDPd) + +For high-impact services (VPN daemons, routing protocols), make effort to +determine if changes can be applied non-disruptively before resorting to restarts. + +Never modify active configuration directly unless absolutely necessary. Instead, +generate configuration files and apply them with a single command like service +reload through systemd. For example, save iptables rules to a file and load them +with ``iptables-restore`` rather than executing iptables commands one by one. + +The ``apply()`` and ``generate()`` functions may raise ``ConfigError`` if the +daemon fails to start with the updated config. However, this is not a substitute +for proper config validation in the ``verify()`` function. Make reasonable effort +to verify that generated configuration is valid and will be accepted by the daemon, +including cross-checks with other VyOS configuration subtrees when necessary. + +Exceptions like ``VyOSError`` (raised by ``vyos.config.Config`` on improper +operations) should not be silenced or caught. While this may produce less polished +error output for users, it generates better bug reports and helps IT professionals +debug issues. + +For reference implementations, see ``ntp.py`` or ``interfaces-bonding.py`` (for +tag nodes) in the vyos-1x_ repository. + +XML Definitions +--------------- + +XML interface definitions define the CLI structure. Before VyOS 1.2 (crux), these +files were created manually. After a redesign process_, new-style templates are +automatically generated from XML input files. + +VyOS interface definitions come with a RelaxNG schema located in the vyos-1x_ +module. This schema is a modified version from VyConf_ (VyOS 2.0). VyOS 1.2.x +interface definitions are reusable in future VyOS versions with minimal changes. + +Schemas provide two benefits: + +* Complete grammar verification +* Automatic validation against the schema + +The `scripts/build-command-templates` script converts XML definitions to +old-style templates and verifies them against the schema. A bad definition +causes the package build to fail. While the XML format is verbose, no other +format provides this level of verification. Specialized XML editors can help +manage verbosity. + +Example XML interface definition: .. code-block:: xml @@ -460,35 +396,34 @@ Example: -Command definitions are purely declarative, and cannot contain any logic. All -logic for generating config files for target applications, restarting services -and so on is implemented in configuration scripts instead. +XML definitions are purely declarative and contain no logic. All logic for +generating config files, restarting services, and related tasks is implemented +in configuration scripts. -GNU Preprocessor ----------------- +Template Processors +------------------- -XML interface definition files use the `xml.in` file extension which was -implemented in :vytask:`T1843`. XML interface definitions tend to have a lot of -duplicated code in areas such as: +XML interface definition files use the ``.xml.in`` file extension (implemented +in :vytask:`T1843`). These files use the GCC preprocessor to reduce code duplication +in common areas: -* VIF (incl. VIF-S/VIF-C) -* Address +* VIF (including VIF-S and VIF-C) +* Address configuration * Description -* Enabled/Disabled +* Enabled/Disabled state -Instead of supplying all those XML nodes multiple times there are now include -files with predefined features. Brief overview: +Instead of repeating XML nodes, use include files with predefined features: -* `IPv4, IPv6 and DHCP(v6)`_ address assignment -* `IPv4, IPv6`_ address assignment +* `IPv4, IPv6, and DHCP(v6)`_ address assignment +* `IPv4 and IPv6`_ address assignment * `VLAN (VIF)`_ definition * `MAC address`_ assignment -All interface definition XML input files (.in suffix) will be sent to the GCC -preprocess and the output is stored in the `build/interface-definitions` -folder. The previously mentioned `scripts/build-command-templates` script -operates on the `build/interface-definitions` folder to generate all required -CLI nodes. +The ``.in`` files are preprocessed and stored in the `build/interface-definitions` +folder. The `scripts/build-command-templates` script then operates on this folder +to generate all required CLI nodes. + +Example preprocessor output: .. code-block:: none @@ -503,63 +438,55 @@ CLI nodes. Generating build/interface-definitions/tftp-server.xml from interface-definitions/tftp-server.xml.in [...] +Command Definition Guidelines +------------------------------ -Guidelines ----------- +Use of Numbers +^^^^^^^^^^^^^^ -Use of numbers -^^^^^^^^^^^^^^^ +Avoid using numbers in command names unless the number is part of a protocol +name or similar. For example, ``protocols ospfv3`` is appropriate, but ``server-1`` +is questionable. -Use of numbers in command names **should** be avoided unless a number is a -part of a protocol name or similar. Thus, ``protocols ospfv3`` is perfectly -fine, but something like ``server-1`` is questionable at best. +Help Strings +^^^^^^^^^^^^ -Help String -^^^^^^^^^^^ +Follow these guidelines for consistent, readable help strings: -To ensure uniform look and feel, and improve readability, we should follow a -set of guidelines consistently. +**Capitalization and Punctuation** -Capitalization and punctuation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +* Capitalize the first word of every help string. +* Do not use a period at the end of help strings. -The first word of every help string **must** be capitalized. There **must not** -be a period at the end of help strings. - -Rationale: this seems to be the unwritten standard in network device CLIs, and -a good aesthetic compromise. +This standard mirrors network device CLIs and improves aesthetics. Examples: * Good: "Frobnication algorithm" * Bad: "frobnication algorithm" * Bad: "Frobnication algorithm." -* Horrible: "frobnication algorithm." +* Incorrect: "frobnication algorithm." -Use of abbreviations and acronyms -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**Abbreviations and Acronyms** -Abbreviations and acronyms **must** be capitalized. +* Capitalize all abbreviations and acronyms. Examples: * Good: "TCP connection timeout" * Bad: "tcp connection timeout" -* Horrible: "Tcp connection timeout" +* Bad: "Tcp connection timeout" -Acronyms also **must** be capitalized to visually distinguish them from normal -words: +* Capitalize acronyms to distinguish them from normal words. Examples: -* Good: RADIUS (as in remote authentication for dial-in user services) -* Bad: radius (unless it's about the distance between a center of a circle and - any of its points) +* Good: RADIUS (remote authentication for dial-in user services) +* Bad: radius (unless referring to circular distance) -Some abbreviations are traditionally written in mixed case. Generally, if it -contains words "over" or "version", the letter **should** be lowercase. If -there's an accepted spelling (especially if defined by an RFC or another -standard), it **must** be followed. +* Follow accepted spelling conventions for mixed-case abbreviations. If it + contains "over" or "version", use lowercase. Follow RFC or standard spellings + when they exist. Examples: @@ -567,48 +494,79 @@ Examples: * Bad: PPPOE, IPSEC * Bad: pppoe, ipsec -Use of verbs -~~~~~~~~~~~~ +**Verbs** -Verbs **should** be avoided. If a verb can be omitted, omit it. +* Avoid verbs. If a verb can be omitted, omit it. Examples: * Good: "TCP connection timeout" * Bad: "Set TCP connection timeout" -If a verb is essential, keep it. For example, in the help text of ``set system -ipv6 disable-forwarding``, "Disable IPv6 forwarding on all interfaces" is a -perfectly justified wording. - -Prefer infinitives -~~~~~~~~~~~~~~~~~~ +* When a verb is essential, use it. For example: "Disable IPv6 forwarding on all interfaces" for ``set system ipv6 disable-forwarding``. -Verbs, when they are necessary, **should** be in their infinitive form. +* Use infinitive form for necessary verbs. Examples: * Good: "Disable IPv6 forwarding" * Bad: "Disables IPv6 forwarding" +vyos-configd Considerations +--------------------------- + +All scripts now run under the config daemon and must conform to these requirements: + +1. The signature and first four lines of ``get_config(...)`` **must** be as specified above. + +2. Each of ``get_config``, ``verify``, ``apply``, and ``generate`` **must** appear + with the correct signatures, even if they are a no-op. + +3. ``Config`` objects other than those in ``get_config`` **must not** appear. + +4. The legacy function ``my_set`` **must not** appear. Modifications to active + config **should not** appear in new code (alternative mechanisms may be used + if absolutely necessary). + +Behind the Scenes +================= + +Template System +--------------- + +VyOS bash (or vbash) completion is defined in templates. Templates are text files +(called ``node.def``) stored in a directory tree where: + +* Directory names define command names +* Template files define command behavior + +Before VyOS 1.2 (crux), templates were created manually. After a redesign process_, +new-style templates are automatically generated from XML input files. + +GNU Preprocessor +^^^^^^^^^^^^^^^^ + +XML interface definition files use the ``.xml.in`` extension. The GCC preprocessor +processes these files to reduce duplication in common elements like VIF, address, +description, and enabled/disabled states. Preprocessed files are stored in +`build/interface-definitions`, where `scripts/build-command-templates` converts +them to CLI nodes. C++ Backend Code ================ -The CLI parser used in VyOS is a mix of bash, bash-completion helper and the -C++ backend library [vyatta-cfg](https://github.com/vyos/vyatta-cfg). This -section is a reference of common CLI commands and the respective entry point -in the C/C++ code. - -* ``set`` +The VyOS CLI parser combines bash, bash-completion helpers, and the C++ backend +library `vyatta-cfg `__. This section +references common CLI commands and their C/C++ entry points: - - https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/cstore/cstore.cpp#L352 - - https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/cstore/cstore.cpp#L2549 +**set**: +* https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/cstore/cstore.cpp#L352 +* https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/cstore/cstore.cpp#L2549 -* ``commit`` +**commit**: - - https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/commit/commit-algorithm.cpp#L1252 +* https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/commit/commit-algorithm.cpp#L1252 .. include:: /_include/common-references.txt From d94214d083aa9a01f5146184723d5506ca5f865f Mon Sep 17 00:00:00 2001 From: Quill Cronwall Date: Sat, 13 Dec 2025 12:02:15 +0900 Subject: [PATCH 10/10] Revise proofreading of development.rst --- docs/contributing/development.rst | 142 ++++++++++++++---------------- 1 file changed, 66 insertions(+), 76 deletions(-) diff --git a/docs/contributing/development.rst b/docs/contributing/development.rst index b4993b6802..107ce4bee7 100644 --- a/docs/contributing/development.rst +++ b/docs/contributing/development.rst @@ -1,3 +1,5 @@ +:lastproofread: 2025-12-12 + .. _development: ########### @@ -8,9 +10,9 @@ Learn how to contribute to VyOS. .. _architecture_overview: -Architecture Overview +Architecture overview ===================== -All VyOS source code is hosted on GitHub in the VyOS organization: +VyOS source code is hosted on GitHub in the VyOS organization: https://github.com/vyos VyOS is composed of multiple modules spread across different @@ -28,7 +30,7 @@ VyOS ISO build scripts are hosted in the `README.md file `__ for more information on building VyOS ISO images. -Contributing Code +Contributing code ================= .. warning:: @@ -42,18 +44,19 @@ All submissions must adhere to these guidelines: * Each commit addresses a single issue or feature. * Each commit message references a Phabricator_ task ID (for example, ``T1234``). -* Each commit includes a username and email to identify the author. +* Each commit is associated with a username and email address + to identify the author (see `Configure your Git identity`_). * Only submit bugfixes in packages other than https://github.com/vyos/vyos-1x. * Commits follow the `coding guidelines`_ outlined below. -Determining Package Ownership +Determining package ownership ----------------------------- To determine which VyOS package contains a file you want to modify, use Debian's ``dpkg -S`` command on your running VyOS installation. -Submitting Your Code +Submitting your code -------------------- Fork the repository and submit a GitHub pull request. This is the preferred way @@ -69,6 +72,8 @@ To fork a VyOS repository: - Clone: ``git clone https://github.com//vyos-1x.git`` - Add remote: ``git remote add myfork https://github.com//vyos-1x.git`` +.. _Configure your Git identity: + 3. Configure your Git identity: .. code-block:: none @@ -95,7 +100,7 @@ maintainers@vyos.net or attach them directly to the Phabricator_ task: * Export last commit: ``git format-patch`` * Export last two commits: ``git format-patch -2`` -Commit Messages +Commit messages =============== For guidance on writing commit messages, review the file history @@ -111,7 +116,7 @@ create a Phabricator_ task first. Reference the task ID in your commit message: If your pull request lacks a Phabricator_ reference, maintainers will request that you amend the commit message. -Writing Good Commit Messages +Writing good commit messages ----------------------------- Follow the format described in `Git documentation `__ @@ -152,7 +157,7 @@ Constraints: New functionality must use the new XML/Python interface, not old-style templates (``node.def`` files and Perl/Bash code). -Coding Guidelines +Coding guidelines ================= VyOS maintains consistent coding standards to help contributors navigate the @@ -173,7 +178,7 @@ to your ``.vimrc`` file: Then use ``gg=G`` in command mode to run the linter. -Text Generation +Text generation --------------- Use a template processor for generating config files: @@ -184,7 +189,7 @@ Use a template processor for generating config files: * Template processors **must** be used for structured, multi-line formats (for example, ISC DHCPd configuration). -Python Code +Python code ----------- Configuration scripts and operation mode scripts written in Python3 should @@ -252,22 +257,22 @@ Structure your scripts with these functions: print(e) sys.exit(1) -**``get_config()``**: This function converts VyOS config to an abstract internal -representation. No other function may call the ``vyos.config.Config`` object -directly. Limiting config reads to one function makes it easier to modify the -config syntax in the future. Additionally, this design improves testability since -you can construct an internal representation by hand rather than mocking the -entire config subsystem. +``get_config()``: This function converts a VyOS config object to an abstract +internal representation. No other function may call the ``vyos.config.Config`` +object directly. Limiting config reads to one function makes it easier to +modify the config syntax in the future. Additionally, this design improves +testability since you can construct an internal representation by hand rather +than mocking the entire config subsystem. -**``verify()``**: This function validates the internal representation. It must +``verify()``: This function validates the internal representation. It must raise ``ConfigError`` with a descriptive message if the config is invalid. It **must not** make any changes to the system. This design enables future features like commit dry-run ("commit test" as in JunOS) where the system can abort a commit before making changes. -**``generate()``**: This function generates config files for system components. +``generate()``: This function generates config files for system components. -**``apply()``**: This function applies the generated configuration to the live +``apply()``: This function applies the generated configuration to the live system. Prefer non-disruptive reload when possible. Disruptive operations like daemon restarts are acceptable only when: @@ -294,17 +299,36 @@ error output for users, it generates better bug reports and helps IT professiona debug issues. For reference implementations, see ``ntp.py`` or ``interfaces-bonding.py`` (for -tag nodes) in the vyos-1x_ repository. +tag nodes) in the `vyos-1x `__ repository. -XML Definitions ---------------- +Other considerations: ``vyos-configd`` +--------------------------- + +All scripts now run under the config daemon and must conform to these requirements: + +1. The signature and first four lines of ``get_config(...)`` **must** be as specified above. + +2. Each of ``get_config``, ``verify``, ``apply``, and ``generate`` **must** appear + with the correct signatures, even if they are a no-op. -XML interface definitions define the CLI structure. Before VyOS 1.2 (crux), these -files were created manually. After a redesign process_, new-style templates are +3. ``Config`` objects other than those in ``get_config`` **must not** appear. + +4. The legacy function ``my_set`` **must not** appear. Modifications to active + config **should not** appear in new code (alternative mechanisms may be used + if absolutely necessary). + +XML for CLI definitions +======================= + +XML interface definitions define the VyOS CLI structure. +Before VyOS ``1.2`` (crux), these +files were created manually. After a redesign, new-style templates are automatically generated from XML input files. -VyOS interface definitions come with a RelaxNG schema located in the vyos-1x_ -module. This schema is a modified version from VyConf_ (VyOS 2.0). VyOS 1.2.x +VyOS interface definitions come with a RelaxNG schema located in the +`vyos-1x `__ +repository. This schema is a modified version from ``VyConf`` (VyOS ``2.0``). +VyOS ``1.2.x`` interface definitions are reusable in future VyOS versions with minimal changes. Schemas provide two benefits: @@ -312,7 +336,8 @@ Schemas provide two benefits: * Complete grammar verification * Automatic validation against the schema -The `scripts/build-command-templates` script converts XML definitions to +The `build-command-templates `__ +script converts XML definitions to old-style templates and verifies them against the schema. A bad definition causes the package build to fail. While the XML format is verbose, no other format provides this level of verification. Specialized XML editors can help @@ -414,14 +439,18 @@ in common areas: Instead of repeating XML nodes, use include files with predefined features: -* `IPv4, IPv6, and DHCP(v6)`_ address assignment -* `IPv4 and IPv6`_ address assignment -* `VLAN (VIF)`_ definition -* `MAC address`_ assignment +* `IPv4, IPv6, and DHCP(v6) `__ + address assignment. +* `IPv4 and IPv6 `__ + address assignment. +* `VLAN (VIF) `__ + definition. +* `MAC address `__ + assignment. -The ``.in`` files are preprocessed and stored in the `build/interface-definitions` -folder. The `scripts/build-command-templates` script then operates on this folder -to generate all required CLI nodes. +The ``.in`` files are preprocessed and stored in the `interface-definitions `__ +folder. The `scripts/build-command-templates `__ +script then operates on this folder to generate all required CLI nodes. Example preprocessor output: @@ -512,45 +541,6 @@ Examples: * Good: "Disable IPv6 forwarding" * Bad: "Disables IPv6 forwarding" -vyos-configd Considerations ---------------------------- - -All scripts now run under the config daemon and must conform to these requirements: - -1. The signature and first four lines of ``get_config(...)`` **must** be as specified above. - -2. Each of ``get_config``, ``verify``, ``apply``, and ``generate`` **must** appear - with the correct signatures, even if they are a no-op. - -3. ``Config`` objects other than those in ``get_config`` **must not** appear. - -4. The legacy function ``my_set`` **must not** appear. Modifications to active - config **should not** appear in new code (alternative mechanisms may be used - if absolutely necessary). - -Behind the Scenes -================= - -Template System ---------------- - -VyOS bash (or vbash) completion is defined in templates. Templates are text files -(called ``node.def``) stored in a directory tree where: - -* Directory names define command names -* Template files define command behavior - -Before VyOS 1.2 (crux), templates were created manually. After a redesign process_, -new-style templates are automatically generated from XML input files. - -GNU Preprocessor -^^^^^^^^^^^^^^^^ - -XML interface definition files use the ``.xml.in`` extension. The GCC preprocessor -processes these files to reduce duplication in common elements like VIF, address, -description, and enabled/disabled states. Preprocessed files are stored in -`build/interface-definitions`, where `scripts/build-command-templates` converts -them to CLI nodes. C++ Backend Code ================ @@ -559,12 +549,12 @@ The VyOS CLI parser combines bash, bash-completion helpers, and the C++ backend library `vyatta-cfg `__. This section references common CLI commands and their C/C++ entry points: -**set**: +``set``: * https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/cstore/cstore.cpp#L352 * https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/cstore/cstore.cpp#L2549 -**commit**: +``commit``: * https://github.com/vyos/vyatta-cfg/blob/0f42786a0b3/src/commit/commit-algorithm.cpp#L1252