vaapi-vulkan-nvidia

A VA-API driver backend for NVIDIA GPUs that uses Vulkan Video for hardware-accelerated encoding.

This driver implements the VA-API interface (libva) on top of NVIDIA's Vulkan Video extensions, enabling VA-API consumers (like FFmpeg) to use NVIDIA hardware encoding and decoding without relying on NVENC/NVDEC directly.

Status

Early development (v0.0.1). Currently only tested on 50xx series, help wanted.

Currently supported:

• H.264 encode
  • Profiles: constrained baseline, main, high
  • Rate control: CQP, CBR, VBR
• H.264 decode (non-HDR)
  • Profiles: constrained baseline, main, high

Architecture

The driver is split into three components:

• x64 VA-API driver (src/x64_vaapi/) — The main driver, a shared library that implements VA-API using Vulkan Video directly.
• i686 VA-API shim (src/i686_vaapi/) — A 32-bit VA-API driver that proxies requests to the x64 host process, enabling 32-bit applications to use the driver.
• x64 proxy host (src/x64_proxyhost/) — A 64-bit daemon that receives proxied requests from the i686 shim and forwards them to the real x64 VA-API driver.

The i686 shim + proxyhost adds some overhead due to IPC on calls involving vaCreateImage, vaDeriveImage .etc via mapped buffers.

Benchmarks encoding Big Buck Bunny 1080p (14315 frames) on a 50xx series GPU:

Bitrate	Arch	FPS	Speed	Size	Penalty
~25 Mbit CBR	x64	376	15.7x	1581 MiB	—
~25 Mbit CBR	i686	274	11.4x	1580 MiB	27%
~5 Mbit CBR	x64	390	16.3x	365 MiB	—
~5 Mbit CBR	i686	344	14.4x	365 MiB	12%

The proxy overhead scales with bitrate: at 25 Mbit the i686 path is ~27% slower and uses ~45% of a CPU core for IPC, dropping to ~12% slower and ~18% of a core at 5 Mbit.

Steam game streaming uses DMA-BUF for transfers from the DRM to to the VAAPI driver. The i686 shimm handles this by passing the fd across a unix socket with SCM_RIGHTS, allowing the proxyhost to read it without copying the data which greatly reduces the overhead for that usecase.

Requirements

• NVIDIA GPU with Vulkan Video encode support
• NVIDIA proprietary drivers
• libva development headers
• Vulkan headers and loader
• pkg-config, gcc, make
• For i686 support: glibc-devel.i686, libva-devel.i686

Building

make            # build all (x64 driver, i686 shim, proxy host)
make x64_vaapi  # build only the x64 driver

Install

sudo make install

This installs:

• nvidia_vulkan_drv_video.so to /usr/lib64/dri/ (x64) and /usr/lib/dri/ (i686)
• nvidia_vulkan_proxy to /usr/lib64/

Docker

A Fedora-based Docker environment is provided for building and testing.

# Build inside container
make docker

# Run a specific target
make docker TARGET=test

# Interactive shell
make shell

Requires the NVIDIA container runtime (--runtime=nvidia).

Testing

# Verify the driver loads
make docker TARGET="test OP=vainfo"

# Short encode test (synthetic input)
make docker TARGET="test OP=encode"

# Decode test (encodes a reference, then decodes with VA-API)
make docker TARGET="test OP=decode"

# Long encode test (Big Buck Bunny, see below)
make docker TARGET="test OP=longencode"

Test variants can be controlled with the following parameters:

Parameter	Values	Default
OP	vainfo, encode, decode, longencode	all
ARCH	x64, i686	all
CODEC	h264	all
RC	cqp, cbr, vbr	all
QP	1–51 (CQP only)	18,26,34,42
DEBUG	x64, i686, proxy

make docker TARGET="test OP=encode ARCH=i686 RC=cbr"

i686 RPMs

The i686 tests require 32-bit builds of ffmpeg and libva-utils which are not available in the standard Fedora repos. These RPMs must be manually downloaded and placed in the rpms/ directory. See rpms/README.md for the full list of required files and download links.

The Docker build will succeed without these RPMs, but i686 tests will not be available.

Long encode input

The longencode tests use Big Buck Bunny as input. Download the video from:

https://download.blender.org/peach/

Place the file at .temp/big_buck_bunny_1080p_stereo.avi.

Usage

Set the VA-API driver name to use this backend:

export LIBVA_DRIVER_NAME=nvidia_vulkan

Then use any VA-API consumer as normal, e.g.:

# Encode
ffmpeg -vaapi_device /dev/dri/renderD128 -i input.mp4 \
    -vf "format=nv12,hwupload" -c:v h264_vaapi output.mp4

# Decode
ffmpeg -vaapi_device /dev/dri/renderD128 \
    -hwaccel vaapi -hwaccel_output_format vaapi \
    -i input.mp4 \
    -vf "hwdownload,format=nv12,format=yuv420p" -c:v ffv1 output.mkv

License

This project is licensed under the MIT License - see the LICENSE file for details.