SNP and vhost-user-fs-pci is not supported #269
Description
Host environment
-
Operating system: Ubuntu Linux 24.04
-
OS/kernel version: 6.12.12
-
Architecture: x86_64 (AMD EPYC 9474F - AMD SEV-SNP)
-
QEMU flavor: qemu-system-x86_64
-
QEMU version: 10.0.0
-
QEMU command line:
sudo ./virtiofsd \ --socket-path /var/run/virtiofs/cfg.sock \ --shared-dir=/tmp \ --sandbox=namespace & sudo ./qemu-system-x86_64 \ -nodefaults \ -enable-kvm \ -cpu host \ -object memory-backend-memfd,id=mem0,size=2048M,share=on \ -object sev-snp-guest,id=sev0,policy=0x30000,cbitpos=51,reduced-phys-bits=1 \ -machine q35,confidential-guest-support=sev0,memory-encryption=sev0,memory-backend=mem0 \ -smp cpus=1 \ -drive file=ubuntu.qcow2,if=none,id=disk0,format=qcow2 \ -device virtio-blk-pci,drive=disk0 \ -device amd-iommu \ -chardev socket,id=fs0,path=/var/run/virtiofs/cfg.sock \ -device vhost-user-fs-pci,chardev=fs0,tag=cfg \ -bios ./ovmf-dist/amdsev/OVMF.fd \ -kernel ./linux-guest-6.12.15-1/boot/vmlinuz-6.12.15-1 \ -initrd ./initrd/initrd.img \ -append 'console=ttyS0' \ -display none \ -nographic \ -chardev stdio,id=stdio0,signal=off \ -serial chardev:stdio0 \ -D /tmp/qemu-vmm.log \ -d 'guest_errors,unimp,trace:virtio*'
Emulated/Virtualized environment
- Operating system: Ubuntu 24.04
- OS/kernel version: 6.12.15
- Architecture: x86_64
Description of problem
Trying to make use of vhost-user-fs-pci with sev-snp-guest enabled doesn't work.
The system reports that vhost-user-fs-pci doesn't support IOMMU but as far as I understand
we need IOMMU for the virtio protocol to fully function.
Steps to reproduce
- Ensure you are running on a system with AMD SNP support:
sudo dmesg | grep -i sev
[ 0.000000] SEV-SNP: RMP table physical range [0x000000bfbd000000 - 0x000000c07d8fffff]
[ 0.003807] SEV-SNP: Reserving start/end of RMP table on a 2MB boundary [0x000000c07d800000]
[ 8.085220] ccp 0000:06:00.5: sev enabled
[ 16.226155] ccp 0000:06:00.5: SEV API:1.55 build:28
[ 16.226162] ccp 0000:06:00.5: SEV-SNP API:1.55 build:28
[ 16.239284] kvm_amd: SEV enabled (ASIDs 15 - 1006)
[ 16.239289] kvm_amd: SEV-ES enabled (ASIDs 1 - 14)
[ 16.239292] kvm_amd: SEV-SNP enabled (ASIDs 1 - 14)
- Use an OVMF which supports AMD SNP: https://github.com/tianocore/edk2.git branch: edk2-stable202502
- Launch the virtiofs daemon process.
- Launch qemu with device
vhost-user-fs-pci - The qemu process will terminate with the following error message:
qemu-system-x86_64: -device vhost-user-fs-pci,chardev=fs0,tag=cfg: iommu_platform=true is not supported by the device
Additional information
It does launch if I disable any AMD SEV-SNP functionality from the VM:
sudo ./qemu-system-x86_64 \
-nodefaults \
-enable-kvm \
-cpu host \
-object memory-backend-memfd,id=mem0,size=2048M,share=on \
-machine q35,memory-backend=mem0 \
-smp cpus=1 \
-drive file=ubuntu.qcow2,if=none,id=disk0,format=qcow2 \
-device virtio-blk-pci,drive=disk0 \
-device amd-iommu \
-chardev socket,id=fs0,path=/var/run/virtiofs/cfg.sock \
-device vhost-user-fs-pci,chardev=fs0,tag=cfg \
-bios ./ovmf-dist/x86_64/OVMF.fd \
-kernel ./linux-guest-6.12.15-1-/boot/vmlinuz-6.12.15-1 \
-initrd ./initrd/initrd.img \
-append 'console=ttyS0' \
-display none
-nographic
-chardev stdio,id=stdio0,signal=off \
-serial chardev:stdio0 \
-D /tmp/qemu-vmm.log \
-d 'guest_errors,unimp,trace:virtio*'
BTW: I've also managed to reproduce the same bug on AMD's fork:
- Repo: https://github.com/AMDESE/qemu.git
- Branch:
snp-latest
Configure flags:
--target-list=x86_64-softmmu \
--prefix=/builder/out/qemu-dist \
--sysconfdir=/builder/out/qemu-dist/etc \
--libdir=/builder/out/qemu-dist/lib \
--libexecdir=/builder/out/qemu-dist/lib/qemu \
--localstatedir=/builder/out/qemu-dist/var \
--ninja=/usr/bin/ninja \
--python=/usr/bin/python3 \
--with-pkgversion=jito-qemu \
--cc=/usr/bin/x86_64-linux-gnu-gcc-13 \
--static \
--disable-cocoa \
--disable-curses \
--disable-dbus-display \
--disable-gtk \
--disable-gtk-clipboard \
--disable-opengl \
--disable-png \
--disable-sdl \
--disable-sdl-image \
--disable-spice \
--disable-spice-protocol \
--disable-virglrenderer \
--disable-vnc \
--disable-vnc-jpeg \
--disable-vnc-sasl \
--disable-vte \
--disable-alsa \
--disable-coreaudio \
--disable-dsound \
--disable-jack \
--disable-oss \
--disable-pa \
--disable-pipewire \
--disable-sndio \
--disable-vvfat \
--disable-vdi \
--disable-qed \
--disable-qcow1 \
--disable-bochs \
--disable-cloop \
--disable-dmg \
--disable-parallels \
--disable-vpc \
--disable-vmdk \
--disable-vhdx \
--disable-bzip2 \
--disable-lzfse \
--disable-snappy \
--disable-lzo \
--disable-netmap \
--disable-l2tpv3 \
--disable-slirp-smbd \
--disable-vde \
--disable-vmnet \
--disable-vhost-user-blk-server \
--disable-vfio-user-server \
--disable-curl \
--disable-glusterfs \
--disable-libiscsi \
--disable-libnfs \
--disable-libssh \
--disable-mpath \
--disable-rbd \
--disable-vduse-blk-export \
--disable-virtfs \
--disable-fuse \
--disable-fuse-lseek \
--disable-blkio \
--disable-nettle \
--disable-gcrypt \
--disable-gnutls \
--disable-crypto-afalg \
--disable-libkeyutils \
--disable-libkeyutils \
--disable-auth-pam \
--disable-keyring \
--disable-selinux \
--disable-u2f \
--disable-brlapi \
--disable-canokey \
--disable-hvf \
--disable-hv-balloon \
--disable-libdaxctl \
--disable-libudev \
--disable-libusb \
--disable-nvmm \
--disable-rdma \
--disable-smartcard \
--disable-usb-redir \
--disable-whpx \
--disable-xen \
--disable-xen-pci-passthrough \
--disable-guest-agent \
--disable-guest-agent-msi \
--disable-colo-proxy \
--disable-rutabaga-gfx \
--disable-vhost-crypto \
--disable-capstone \
--disable-docs \
--disable-gettext \
--disable-iconv \
--disable-libdw \
--disable-pixman \
--disable-sparse \
--disable-xkbcommon \
--disable-attr \
--disable-gio \
--disable-multiprocess \
--disable-plugins \
--disable-qpl \
--disable-replication \
--disable-uadk \
--disable-libvduse \
--disable-libpmem \
--disable-user \
--disable-bsd-user \
--disable-linux-user \
--disable-tcg \
--disable-debug-tcg \
--disable-tcg-interpreter \
--disable-hexagon-idef-parser \
--disable-qom-cast-debug \
--enable-kvm \
--enable-system \
--enable-pie \
--enable-lto \
--enable-af-xdp \
--enable-slirp \
--enable-vhost-kernel \
--enable-vhost-net \
--enable-vhost-user \
--enable-vhost-vdpa \
--enable-bpf \
--enable-coroutine-pool \
--enable-linux-aio \
--enable-linux-io-uring \
--enable-malloc-trim \
--enable-membarrier \
--enable-cap-ng \
--enable-seccomp \
--enable-stack-protector \
--enable-tpm \
--enable-zstd \
--enable-numa \
--enable-fdt=disabled \
--enable-install-blobs \
--enable-tools \
--enable-trace-backends=log \
--enable-strip \
--x86-version=4 \
--extra-cflags=-O2 -fno-semantic-interposition -fdevirtualize-at-ltrans -flto=auto -fuse-linker-plugin -falign-functions=32 -D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations -Wno-error=stringop-overflow -Wformat -Werror=format-security -Werror=implicit-function-declaration -fstack-protector-strong -fstack-clash-protection -fcf-protection -fipa-pta \
--extra-ldflags=-Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-O1 -Wl,--as-needed