From 6638bc306a0af010d4301783f05685670b1037a6 Mon Sep 17 00:00:00 2001 From: Michael Smithhisler Date: Mon, 1 Dec 2025 16:35:13 -0500 Subject: [PATCH 1/4] add new config parameters to nomad qemu driver --- .../v1.11.x/content/docs/deploy/task-driver/qemu.mdx | 4 +++- .../content/docs/job-declare/task-driver/qemu.mdx | 11 +++++++++-- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx b/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx index 42d1f91eec..37908d8fd3 100644 --- a/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx +++ b/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx @@ -44,7 +44,7 @@ The `qemu` driver will set the following client attributes: - `driver.qemu` - Set to `1` if QEMU is found on the host node. Nomad determines this by executing `qemu-system-x86_64 -version` on the host and parsing the output -- `driver.qemu.version` - Version of `qemu-system-x86_64`, ex: `2.4.0` +- `driver.qemu.version` - Version of `qemu.config.fingerprint_emulator`, ex: `2.4.0` Here is an example of using these properties in a job file: @@ -78,6 +78,8 @@ plugin "qemu" { including flags that provide the VM with access to host devices such as USB drives. Refer to the [QEMU documentation] for the available flags. +- `fingerprint_emulator` (`string`: `qemu-system-x86_64`) - Specifies + the emulator binary used to fingerprint QEMU on the Nomad client. ## Resource Isolation diff --git a/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx b/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx index 2eee6c2019..4f4596d763 100644 --- a/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx +++ b/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx @@ -15,7 +15,8 @@ host machine to the guest virtual machine, and provides configuration for resource allocation. The `qemu` driver can execute any regular `qemu` image (e.g. `qcow`, `img`, -`iso`), and is currently invoked with `qemu-system-x86_64`. +`iso`), and is invoked with the QEMU binary specified by the `emulator` +parameter. The driver requires the image to be accessible from the Nomad client via the [`artifact` downloader](/nomad/docs/job-specification/artifact). @@ -31,6 +32,7 @@ task "webservice" { config { image_path = "/path/to/my/linux.img" + machine_type = "q35" accelerator = "kvm" graceful_shutdown = true args = ["-nodefaults", "-nodefconfig"] @@ -43,7 +45,12 @@ The `qemu` driver supports the following configuration in the job spec: - `image_path` - The path to the downloaded image. In most cases this will just be the name of the image. However, if the supplied artifact is an archive that contains the image in a subfolder, the path will need to be the relative path - (`subdir/from_archive/my.img`). + (`subdir/from_archive/my.img`). This image is given an ID of "image0". + +- `emulator` `(string: "qemu-system-x86_64")` - The QEMU emulator binary used to + run the virtual machine. + +- `machine_type` `(string: "pc")` - Used to select the emulated machine by name. - `drive_interface` - (Optional) This option defines on which type of interface the drive is connected. Available types are: `ide`, `scsi`, `sd`, `mtd`, From 5614aa8ab6bfe2feabe70a46cc61a8e5b794aaac Mon Sep 17 00:00:00 2001 From: Michael Smithhisler Date: Tue, 2 Dec 2025 08:50:38 -0500 Subject: [PATCH 2/4] add upgrade note for QEMU driver update --- .../v1.11.x/content/docs/upgrade/upgrade-specific.mdx | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx b/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx index 81a90bda07..8294aa896c 100644 --- a/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx +++ b/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx @@ -12,6 +12,16 @@ upgrade. However, specific versions of Nomad may have more details provided for their upgrades as a result of new features or changed behavior. This page is used to document those details separately from the standard upgrade flow. +## Nomad 1.11.1 + +In Nomad 1.11.1, `emulator` and `machine_type` were added to the task config, +and `fingerprint_emulator` was added to the driver config. These default to the +previously used values of `qemu-system-x86_64`, and `pc`. Previously, when using +the `kvm` accelerator, the machine type `host` was forced. This is no longer true, +the value for `machine_type` will be used. Additionally, if using resources.cores, +with the `kvm` accelerator, the `-smp` was hardcoded to that number of cores. This +is now only done if the user has not specified a custom -smp flag. + ## Nomad 1.11.0 #### Sysbatch jobs will no longer accept `reschedule` blocks From 6abcd3d471492ebad9a9f03c9c41432e36dc203e Mon Sep 17 00:00:00 2001 From: Michael Smithhisler Date: Tue, 2 Dec 2025 17:04:06 -0500 Subject: [PATCH 3/4] update docs after PR refactor --- .../content/docs/deploy/task-driver/qemu.mdx | 25 +++++++++++++++---- .../docs/job-declare/task-driver/qemu.mdx | 7 +++--- .../content/docs/upgrade/upgrade-specific.mdx | 16 ++++++------ 3 files changed, 33 insertions(+), 15 deletions(-) diff --git a/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx b/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx index 37908d8fd3..d55b56274e 100644 --- a/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx +++ b/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx @@ -43,8 +43,10 @@ way to retrieve the image being run. The `qemu` driver will set the following client attributes: - `driver.qemu` - Set to `1` if QEMU is found on the host node. Nomad determines - this by executing `qemu-system-x86_64 -version` on the host and parsing the output -- `driver.qemu.version` - Version of `qemu.config.fingerprint_emulator`, ex: `2.4.0` + this by executing the first found qemu binary with `--version` on the client and + parsing the output. Nomad does not get the version of each individual qemu emulator. +- `driver.qemu.version` - Version found running the command for `driver.qemu`, ex: `2.4.0` +- `driver.qemu.emulators` - A comma separated list of emulators detected on the client. Here is an example of using these properties in a job file: @@ -52,13 +54,24 @@ Here is an example of using these properties in a job file: job "docs" { # Only run this job where the qemu version is higher than 1.2.3. constraint { - attribute = "${driver.qemu.version}" + attribute = "${attr.driver.qemu.version}" operator = ">" value = "1.2.3" } } ``` +```hcl +job "docs" { + # Only run this job where the the qemu-system-aarch64 emulator was detected. + constraint { + attribute = "${attr.driver.qemu.emulators}" + operator = "set_contains" + value = "aarch64" + } +} +``` + ## Plugin Options ```hcl @@ -78,8 +91,10 @@ plugin "qemu" { including flags that provide the VM with access to host devices such as USB drives. Refer to the [QEMU documentation] for the available flags. -- `fingerprint_emulator` (`string`: `qemu-system-x86_64`) - Specifies - the emulator binary used to fingerprint QEMU on the Nomad client. +- `emulators_allowlist` (`[]string`: `[]`) - Specifies the allowed emulators + tasks may use. When specified Nomad will only fingerprint this list, and + will not allow tasks to run using other emulators. Defaults to fingerprinting + and allowing all emulators. ## Resource Isolation diff --git a/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx b/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx index 4f4596d763..dfe887314b 100644 --- a/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx +++ b/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx @@ -15,8 +15,8 @@ host machine to the guest virtual machine, and provides configuration for resource allocation. The `qemu` driver can execute any regular `qemu` image (e.g. `qcow`, `img`, -`iso`), and is invoked with the QEMU binary specified by the `emulator` -parameter. +`iso`). Nomad uses the QEMU binary specified by the `emulator` +parameter (ex. If `x86_64` is specified, Nomad will use `qemu-system-x86_64`). The driver requires the image to be accessible from the Nomad client via the [`artifact` downloader](/nomad/docs/job-specification/artifact). @@ -32,6 +32,7 @@ task "webservice" { config { image_path = "/path/to/my/linux.img" + emulator = "x86_64" machine_type = "q35" accelerator = "kvm" graceful_shutdown = true @@ -47,7 +48,7 @@ The `qemu` driver supports the following configuration in the job spec: contains the image in a subfolder, the path will need to be the relative path (`subdir/from_archive/my.img`). This image is given an ID of "image0". -- `emulator` `(string: "qemu-system-x86_64")` - The QEMU emulator binary used to +- `emulator` `(string: "x86_64")` - The architecture of the QEMU emulator used to run the virtual machine. - `machine_type` `(string: "pc")` - Used to select the emulated machine by name. diff --git a/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx b/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx index 8294aa896c..ec8982cc8d 100644 --- a/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx +++ b/content/nomad/v1.11.x/content/docs/upgrade/upgrade-specific.mdx @@ -14,13 +14,15 @@ used to document those details separately from the standard upgrade flow. ## Nomad 1.11.1 -In Nomad 1.11.1, `emulator` and `machine_type` were added to the task config, -and `fingerprint_emulator` was added to the driver config. These default to the -previously used values of `qemu-system-x86_64`, and `pc`. Previously, when using -the `kvm` accelerator, the machine type `host` was forced. This is no longer true, -the value for `machine_type` will be used. Additionally, if using resources.cores, -with the `kvm` accelerator, the `-smp` was hardcoded to that number of cores. This -is now only done if the user has not specified a custom -smp flag. +In Nomad 1.11.1, `emulator` and `machine_type` were added to the task config. +These default to the previously used values of `qemu-system-x86_64`, and `pc`. +The `emulators_allowlist` driver config was also added, which defaults to +allowing any emulator to run. + +Previously, when using the `kvm` accelerator the machine type `host` was required. +This is no longer enforced. The value for `machine_type` will be used. Also, if using +resources.cores with the `kvm` accelerator, the `-smp` was hardcoded to resources.cores +value. This is now only done if the user has not specified a custom -smp flag. ## Nomad 1.11.0 From 8039661650d747080d83fddc544769b4d8a8397d Mon Sep 17 00:00:00 2001 From: Aimee Ukasick Date: Fri, 5 Dec 2025 10:38:15 -0600 Subject: [PATCH 4/4] style guide updates --- .../content/docs/deploy/task-driver/qemu.mdx | 14 ++++++++------ .../content/docs/job-declare/task-driver/qemu.mdx | 14 +++++++------- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx b/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx index d55b56274e..d9c1fa8abd 100644 --- a/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx +++ b/content/nomad/v1.11.x/content/docs/deploy/task-driver/qemu.mdx @@ -43,9 +43,11 @@ way to retrieve the image being run. The `qemu` driver will set the following client attributes: - `driver.qemu` - Set to `1` if QEMU is found on the host node. Nomad determines - this by executing the first found qemu binary with `--version` on the client and - parsing the output. Nomad does not get the version of each individual qemu emulator. -- `driver.qemu.version` - Version found running the command for `driver.qemu`, ex: `2.4.0` + this by executing the first found QEMU binary with `--version` on the client + and parsing the output. Nomad does not get the version of each individual QEMU + emulator. +- `driver.qemu.version` - Version found running the command for `driver.qemu`; + for example: `2.4.0` - `driver.qemu.emulators` - A comma separated list of emulators detected on the client. Here is an example of using these properties in a job file: @@ -92,9 +94,9 @@ plugin "qemu" { as USB drives. Refer to the [QEMU documentation] for the available flags. - `emulators_allowlist` (`[]string`: `[]`) - Specifies the allowed emulators - tasks may use. When specified Nomad will only fingerprint this list, and - will not allow tasks to run using other emulators. Defaults to fingerprinting - and allowing all emulators. + tasks may use. When specified, Nomad only fingerprints this list and does not + allow tasks to run using other emulators. Defaults to fingerprinting and + allowing all emulators. ## Resource Isolation diff --git a/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx b/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx index dfe887314b..f415fdcd1a 100644 --- a/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx +++ b/content/nomad/v1.11.x/content/docs/job-declare/task-driver/qemu.mdx @@ -14,9 +14,9 @@ great performance. Currently the `qemu` driver can map a set of ports from the host machine to the guest virtual machine, and provides configuration for resource allocation. -The `qemu` driver can execute any regular `qemu` image (e.g. `qcow`, `img`, -`iso`). Nomad uses the QEMU binary specified by the `emulator` -parameter (ex. If `x86_64` is specified, Nomad will use `qemu-system-x86_64`). +The `qemu` driver can execute any regular `qemu` image, such as `qcow`, `img`, +and `iso`. Nomad uses the QEMU binary specified by the `emulator` parameter. For +example, if you specify `x86_64d, Nomad uses `qemu-system-x86_64`. The driver requires the image to be accessible from the Nomad client via the [`artifact` downloader](/nomad/docs/job-specification/artifact). @@ -43,10 +43,10 @@ task "webservice" { The `qemu` driver supports the following configuration in the job spec: -- `image_path` - The path to the downloaded image. In most cases this will just - be the name of the image. However, if the supplied artifact is an archive that - contains the image in a subfolder, the path will need to be the relative path - (`subdir/from_archive/my.img`). This image is given an ID of "image0". +- `image_path` - The path to the downloaded image. In most cases this is the + name of the image. However, if the supplied artifact is an archive that + contains the image in a subfolder, the path needs to be the relative path + (`subdir/from_archive/my.img`). Nomad gives this image an ID of "image0". - `emulator` `(string: "x86_64")` - The architecture of the QEMU emulator used to run the virtual machine.