Support for libkrun using krunkit
#4137
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirs
reviewed
Oct 4, 2025
afbjorklund
reviewed
Oct 5, 2025
|
@lima-vm/maintainers @lima-vm/committers Any help is appreciated! What am I doing wrong here that an IP is not getting assigned for the For # limactl start default --vm-type=qemu --network=lima:shared
➜ lima2 git:(driver/krun) ✗ arp -an | grep bridge100
? (192.168.105.1) at 52:ed:3c:d4:1e:64 on bridge100 ifscope permanent [bridge]
? (192.168.105.2) at 52:55:55:e0:48:67 on bridge100 ifscope [bridge] # VM IP
? (224.0.0.251) at 1:0:5e:0:0:fb on bridge100 ifscope permanent [ethernet]For ➜ limactl start default --vm-type=krunkit --network=lima:shared
WARN[0000] Template locator "template://_images/ubuntu" should be written "template:_images/ubuntu" since Lima v2.0
WARN[0000] Template locator "template://_default/mounts" should be written "template:_default/mounts" since Lima v2.0
? Creating an instance "default" Proceed with the current configuration
INFO[0001] Starting socket_vmnet daemon for "shared" network
INFO[0001] Starting the instance "default" with external VM driver "krunkit"
INFO[0001] Attempting to download the image arch=aarch64 digest="sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8" location="https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
INFO[0001] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/eccac025a7a4709721a97d4ef1d5973546c85062917f188cdc6d247b665cbab5/data"
INFO[0004] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:2f98346ed3dcaf47cfd6461a5685e582284329a1ece1d952ade881b9fe7491e8" location="https://github.com/containerd/nerdctl/releases/download/v2.1.6/nerdctl-full-2.1.6-linux-arm64.tar.gz"
INFO[0004] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/d82077f1308e8d063805c4c1f761e1c3fd636e18cab7e25919a18e7a0838afa2/data"
INFO[0006] [hostagent] hostagent socket created at /Users/ansumansahoo/.lima/default/ha.sock
INFO[0006] SSH Local Port: 55283
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
...
INFO[0598] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0598] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
FATA[0604] did not receive an event with the "running" status
➜ lima2 git:(driver/krun) ✗ arp -an | grep bridge100
? (192.168.105.1) at 52:ed:3c:d4:1e:64 on bridge100 ifscope permanent [bridge]
? (224.0.0.251) at 1:0:5e:0:0:fb on bridge100 ifscope permanent [ethernet]Also tried this but didn't work: https://github.com/lima-vm/socket_vmnet?tab=readme-ov-file#ip-address-is-not-assigned Is it because the Logs for
|
nirs
reviewed
Oct 13, 2025
arp is not a good way to find the ip, it can take a while until a linux vm is available via arp.
This is not needed since macOS 15, I guess you are on macOS 15 or 26?
The guest agent should not affect this. The vm should get an ip address in few seconds after it starts. Typical when it does not work it is because an issue in vmnet/socketlifterfw,bootpd/kernel. Unfortunately we have very little visibility on the internals.
Good, we know that the vm started.
Looks good
Looks good if I remember correctly. Stuff you can try when you don't get ip address:
If nothing else works, usually reboot fixes the issue. |
|
I tested with socket_vment installed as launchd daemon - I prefer to use a proper daemon instead of manage the daemon using lima with sudo. Using this yaml: And nothing happens - looks like logging errors on startup is broken. I did not try to look at logs since I'm not interested in "lima: shared" mode. After changing the network to: lima starts: And krunkit is running as: This cannot work since lima does not create the socket: And krunkit panics: We need fix krunkit the check if the path exist and fail in a cleaner way. Checking the qemu driver network code I found that we use the wrong Mac address for the device - the Mac address of the user network instead of the socket_vmnet network. I fixed the networking arguments using: diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index db930539..c200f4be 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -17,7 +17,6 @@ import (
"github.com/lima-vm/lima/v2/pkg/iso9660util"
"github.com/lima-vm/lima/v2/pkg/limatype"
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
- "github.com/lima-vm/lima/v2/pkg/limayaml"
"github.com/lima-vm/lima/v2/pkg/networks"
)
@@ -52,6 +51,8 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
var args []string
+
+ // We support only socket_vment - check if we can use it.
nwCfg, err := networks.LoadConfig()
if err != nil {
return nil, err
@@ -60,21 +61,32 @@ func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
if err != nil {
return nil, err
}
- if socketVMNetOk {
- sock, err := networks.Sock(networks.ModeShared)
- if err != nil {
- return nil, err
+ if !socketVMNetOk {
+ return args, errors.New("socket_vmnet is not installed")
+ }
+
+ // Add one or more network devices.
+ for _, nw := range inst.Networks {
+ var sock string
+ if nw.Lima == networks.ModeShared || nw.Lima == networks.ModeBridged {
+ sock, err = networks.Sock(nw.Lima)
+ if err != nil {
+ return nil, err
+ }
+ } else if nw.Socket != "" {
+ sock = nw.Socket
+ } else {
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
}
- networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s",
- sock,
- limayaml.MACAddress(inst.Dir),
- )
- args = append(args, "--device", networkArg)
+ device := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s", sock, nw.MACAddress)
+ args = append(args, "--device", device)
+ }
- return args, nil
+ if len(args) == 0 {
+ return args, fmt.Errorf("no socket_vment networks defined")
}
- return args, errors.New("socket_vmnet is not installed")
+ return args, nil
}
func EnsureDisk(ctx context.Context, inst *limatype.Instance) error {With this krunkit is started with the right arguments: But it takes about a minute until serial.log shows the login prompt, and krunkit does not get an ip address. The issue is that we generate this network_config, which cannot work: krunkit does not have a user network and we create only one virtio-net device. I'm not sure why this configuration does not work, since we don't have any device matching I found that I can remove the user network with this change: diff --git a/pkg/cidata/cidata.go b/pkg/cidata/cidata.go
index 888e17c5..dd0bdd0d 100644
--- a/pkg/cidata/cidata.go
+++ b/pkg/cidata/cidata.go
@@ -257,7 +257,11 @@ func templateArgs(ctx context.Context, bootScripts bool, instDir, name string, i
})
}
- args.Networks = append(args.Networks, Network{MACAddress: limayaml.MACAddress(instDir), Interface: networks.SlirpNICName, Metric: 200})
+ // XXX Should query the driver instead of hardcodeing the name.
+ if *instConfig.VMType != "krunkit" {
+ args.Networks = append(args.Networks, Network{MACAddress: limayaml.MACAddress(instDir), Interface: networks.SlirpNICName, Metric: 200})
+ }
+
for i, nw := range instConfig.Networks {
if i == firstUsernetIndex {
continueWith this change krunkit starts in few seconds and we can login: % ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=error -l nir 192.168.105.2 -i ~/.lima/_config/user
Welcome to Ubuntu 25.04 (GNU/Linux 6.14.0-23-generic aarch64)
...
nir@lima-krunkit:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: lima0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:eb:bf:61 brd ff:ff:ff:ff:ff:ff
altname enx525555ebbf61
inet 192.168.105.2/24 metric 100 brd 192.168.105.255 scope global dynamic lima0
valid_lft 334sec preferred_lft 334sec
inet6 fe80::5055:55ff:feeb:bf61/64 scope link proto kernel_ll
valid_lft forever preferred_lft foreverBut limactl is stuck trying to access ssh via the user network which krunkit does not have: To integrate with krunkit we need to change lima to support driver without a user network:
|
Why can't we have a user network ? |
This comment was marked as off-topic.
This comment was marked as off-topic.
Actually we can! But the result is:
So having an option to run an instance without a user network is better. PatchWith this patch on top of this PR I could start in plain mode: diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index db930539..d86a0c39 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -19,6 +19,7 @@ import (
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
"github.com/lima-vm/lima/v2/pkg/limayaml"
"github.com/lima-vm/lima/v2/pkg/networks"
+ "github.com/lima-vm/lima/v2/pkg/networks/usernet"
)
const (
@@ -52,29 +53,55 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
var args []string
- nwCfg, err := networks.LoadConfig()
- if err != nil {
- return nil, err
- }
- socketVMNetOk, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet)
- if err != nil {
- return nil, err
- }
- if socketVMNetOk {
- sock, err := networks.Sock(networks.ModeShared)
- if err != nil {
- return nil, err
+
+ // Add one or more network devices.
+ for _, nw := range inst.Networks {
+ var sock string
+ var mac string
+
+ if nw.Lima != "" {
+ nwCfg, err := networks.LoadConfig()
+ if err != nil {
+ return nil, err
+ }
+ switch nw.Lima {
+ case networks.ModeUserV2:
+ sock, err = usernet.Sock(nw.Lima, usernet.QEMUSock)
+ if err != nil {
+ return nil, err
+ }
+ mac = limayaml.MACAddress(inst.Dir)
+ case networks.ModeShared, networks.ModeBridged:
+ socketVMNetOk, err := nwCfg.IsDaemonInstalled(networks.SocketVMNet)
+ if err != nil {
+ return nil, err
+ }
+ if !socketVMNetOk {
+ return args, errors.New("socket_vmnet is not installed")
+ }
+ sock, err = networks.Sock(nw.Lima)
+ if err != nil {
+ return nil, err
+ }
+ mac = nw.MACAddress
+ default:
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
+ }
+ } else if nw.Socket != "" {
+ sock = nw.Socket
+ mac = nw.MACAddress
+ } else {
+ return nil, fmt.Errorf("invalid network spec %+v", nw)
}
- networkArg := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s",
- sock,
- limayaml.MACAddress(inst.Dir),
- )
- args = append(args, "--device", networkArg)
+ device := fmt.Sprintf("virtio-net,type=unixstream,path=%s,mac=%s", sock, mac)
+ args = append(args, "--device", device)
+ }
- return args, nil
+ if len(args) == 0 {
+ return args, fmt.Errorf("no socket_vment networks defined")
}
- return args, errors.New("socket_vmnet is not installed")
+ return args, nil
}
func EnsureDisk(ctx context.Context, inst *limatype.Instance) error {
diff --git a/pkg/driver/krunkit/krunkit_driver_darwin.go b/pkg/driver/krunkit/krunkit_driver_darwin.go
index bf5d0a2c..a4bfa7c3 100644
--- a/pkg/driver/krunkit/krunkit_driver_darwin.go
+++ b/pkg/driver/krunkit/krunkit_driver_darwin.go
@@ -19,6 +19,8 @@ import (
"github.com/lima-vm/lima/v2/pkg/executil"
"github.com/lima-vm/lima/v2/pkg/limatype"
"github.com/lima-vm/lima/v2/pkg/limatype/filenames"
+ "github.com/lima-vm/lima/v2/pkg/limayaml"
+ "github.com/lima-vm/lima/v2/pkg/networks/usernet"
"github.com/lima-vm/lima/v2/pkg/ptr"
)
@@ -86,7 +88,15 @@ func (l *LimaKrunDriver) Start(ctx context.Context) (chan error, error) {
}()
l.krunkitWaitCh <- krunCmd.Wait()
}()
-
+ go func() {
+ if usernetIndex := limayaml.FirstUsernetIndex(l.Instance.Config); usernetIndex != -1 {
+ client := usernet.NewClientByName(l.Instance.Config.Networks[usernetIndex].Lima)
+ err := client.ConfigureDriver(ctx, l.Instance, l.SSHLocalPort)
+ if err != nil {
+ l.krunkitWaitCh <- err
+ }
+ }
+ }()
return l.krunkitWaitCh, nil
}
ConfigUsing this config: vmType: krunkit
images:
- location: "https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
arch: "aarch64"
digest: "sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8"
- location: https://cloud-images.ubuntu.com/releases/plucky/release/ubuntu-25.04-server-cloudimg-arm64.img
arch: aarch64
plain: true
portForwards:
- static: true
guestPort: 5201
proto: tcp
networks:
# Required for boot
- lima: user-v2
# Make it accesisble from host and other instances
- socket: /var/run/socket_vmnet
metric: 300 # prefer user-v2lima: user-v2 is required for the ssh connection with current lima. We must change the metric on the socket network to make user-v2 preferred. Setting metric on the lima network does not work. Without I don't have network access from the instance. Testing krunkit driverStarting: % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with external VM driver "krunkit"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] SSH Local Port: 51680
INFO[0000] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0010] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0010] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0010] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0011] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0011] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0011] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell. Lima processes: % ps | grep lima
21492 ttys005 0:00.01 grep lima
21334 ttys006 0:20.70 /Users/nir/src/lima/_output/bin/limactl usernet -p /Users/nir/.lima/_networks/user-v2/usernet_user-v2.pid -e /Users/nir/.lima/_networks/user-v2/user-v2_ep.sock --listen-qemu /Users/nir/.lima/_networks/user-v2/user-v2_qemu.sock --listen /Users/nir/.lima/_networks/user-v2/user-v2_fd.sock --subnet 192.168.104.0/24 --leases 192.168.104.1=52:55:55:f3:38:6a,192.168.104.2=5a:94:ef:e4:0c:dd,192.168.104.3=52:55:55:7d:08:f5,192.168.104.4=52:55:55:92:cc:15
21336 ttys006 0:00.08 /Users/nir/src/lima/_output/bin/limactl hostagent --pidfile /Users/nir/.lima/krunkit/ha.pid --socket /Users/nir/.lima/krunkit/ha.sock krunkit
21340 ttys006 0:00.03 /Users/nir/src/lima/_output/libexec/lima/lima-driver-krunkit
21345 ttys006 0:29.40 krunkit --memory 2048 --cpus 4 --device virtio-serial,logFilePath=/Users/nir/.lima/krunkit/serial.log --krun-log-level 3 --restful-uri none:// --device virtio-blk,path=/Users/nir/.lima/krunkit/diffdisk,format=raw --device virtio-blk,path=/Users/nir/.lima/krunkit/cidata.iso --device virtio-net,type=unixstream,path=/Users/nir/.lima/_networks/user-v2/user-v2_qemu.sock,mac=52:55:55:92:cc:15 --device virtio-net,type=unixstream,path=/var/run/socket_vmnet,mac=52:55:55:12:88:1fAccessing via limactl shell % ./_output/bin/limactl shell krunkit
nir@lima-krunkit:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:92:cc:15 brd ff:ff:ff:ff:ff:ff
altname enx52555592cc15
inet 192.168.104.4/24 metric 200 brd 192.168.104.255 scope global dynamic eth0
valid_lft 3512sec preferred_lft 3512sec
inet6 fe80::5055:55ff:fe92:cc15/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
3: lima1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:55:55:12:88:1f brd ff:ff:ff:ff:ff:ff
altname enx52555512881f
inet 192.168.105.3/24 metric 300 brd 192.168.105.255 scope global dynamic lima1
valid_lft 512sec preferred_lft 512sec
inet6 fe80::5055:55ff:fe12:881f/64 scope link proto kernel_ll
valid_lft forever preferred_lft forever
nir@lima-krunkit:~$ ip r
default via 192.168.104.2 dev eth0 proto dhcp src 192.168.104.4 metric 200
default via 192.168.105.1 dev lima1 proto dhcp src 192.168.105.3 metric 300
192.168.104.0/24 dev eth0 proto kernel scope link src 192.168.104.4 metric 200
192.168.104.2 dev eth0 proto dhcp scope link src 192.168.104.4 metric 200
192.168.105.0/24 dev lima1 proto kernel scope link src 192.168.105.3 metric 300
192.168.105.1 dev lima1 proto dhcp scope link src 192.168.105.3 metric 300 Testing socket_vmnet: % iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51554 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 479 MBytes 4.00 Gbits/sec
[ 5] 1.01-2.01 sec 466 MBytes 3.91 Gbits/sec
[ 5] 2.01-3.00 sec 474 MBytes 3.99 Gbits/sec
[ 5] 3.00-4.01 sec 481 MBytes 4.02 Gbits/sec
[ 5] 4.01-5.01 sec 483 MBytes 4.05 Gbits/sec
[ 5] 5.01-6.01 sec 484 MBytes 4.06 Gbits/sec
[ 5] 6.01-7.01 sec 470 MBytes 3.94 Gbits/sec
[ 5] 7.01-8.01 sec 470 MBytes 3.95 Gbits/sec
[ 5] 8.01-9.01 sec 464 MBytes 3.89 Gbits/sec
[ 5] 9.01-10.01 sec 478 MBytes 4.01 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 4.64 GBytes 3.98 Gbits/sec sender
[ 5] 0.00-10.01 sec 4.64 GBytes 3.98 Gbits/sec receiverTesting user-v2: % iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51691 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 266 MBytes 2.23 Gbits/sec
[ 7] 1.00-2.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 2.00-3.00 sec 259 MBytes 2.17 Gbits/sec
[ 7] 3.00-4.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 4.00-5.00 sec 264 MBytes 2.21 Gbits/sec
[ 7] 5.00-6.00 sec 280 MBytes 2.34 Gbits/sec
[ 7] 6.00-7.00 sec 259 MBytes 2.18 Gbits/sec
[ 7] 7.00-8.00 sec 260 MBytes 2.18 Gbits/sec
[ 7] 8.00-9.00 sec 263 MBytes 2.21 Gbits/sec
[ 7] 9.00-10.00 sec 259 MBytes 2.17 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 2.57 GBytes 2.20 Gbits/sec sender
[ 7] 0.00-10.01 sec 2.56 GBytes 2.20 Gbits/sec receiverTesting multiple instancesI created another instance with same configuration without static port forwarding: vmType: krunkit
images:
- location: "https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
arch: "aarch64"
digest: "sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8"
- location: https://cloud-images.ubuntu.com/releases/plucky/release/ubuntu-25.04-server-cloudimg-arm64.img
arch: aarch64
plain: true
networks:
- lima: user-v2
- socket: /var/run/socket_vmnet
metric: 300Running iperf3 on second instance (krunkit2) connecting to first (krunkit) via the instance IP (socket_vment) or the internal DNS name on the user-v2 network. nir@lima-krunkit2:~$ iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.4 port 54540 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 297 MBytes 2.49 Gbits/sec 0 3.95 MBytes
[ 5] 1.00-2.00 sec 304 MBytes 2.55 Gbits/sec 0 3.95 MBytes
[ 5] 2.00-3.00 sec 308 MBytes 2.59 Gbits/sec 0 3.95 MBytes
[ 5] 3.00-4.00 sec 306 MBytes 2.57 Gbits/sec 0 3.95 MBytes
[ 5] 4.00-5.00 sec 307 MBytes 2.57 Gbits/sec 0 3.95 MBytes
[ 5] 5.00-6.00 sec 310 MBytes 2.60 Gbits/sec 0 3.95 MBytes
[ 5] 6.00-7.00 sec 307 MBytes 2.58 Gbits/sec 0 3.95 MBytes
[ 5] 7.00-8.00 sec 304 MBytes 2.55 Gbits/sec 0 3.95 MBytes
[ 5] 8.00-9.00 sec 309 MBytes 2.59 Gbits/sec 0 3.95 MBytes
[ 5] 9.00-10.00 sec 306 MBytes 2.57 Gbits/sec 0 3.95 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.99 GBytes 2.57 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 2.99 GBytes 2.56 Gbits/sec receiver
nir@lima-krunkit2:~$ iperf3 -c lima-krunkit.internal
Connecting to host lima-krunkit.internal, port 5201
[ 5] local 192.168.104.5 port 37324 connected to 192.168.104.4 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 314 MBytes 2.63 Gbits/sec 0 4.00 MBytes
[ 5] 1.00-2.00 sec 319 MBytes 2.67 Gbits/sec 0 4.00 MBytes
[ 5] 2.00-3.00 sec 316 MBytes 2.65 Gbits/sec 0 4.00 MBytes
[ 5] 3.00-4.00 sec 315 MBytes 2.64 Gbits/sec 0 4.00 MBytes
[ 5] 4.00-5.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 5.00-6.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 6.00-7.00 sec 317 MBytes 2.66 Gbits/sec 0 4.21 MBytes
[ 5] 7.00-8.00 sec 319 MBytes 2.68 Gbits/sec 0 4.21 MBytes
[ 5] 8.00-9.00 sec 316 MBytes 2.65 Gbits/sec 0 4.21 MBytes
[ 5] 9.00-10.00 sec 315 MBytes 2.64 Gbits/sec 0 4.21 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 3.09 GBytes 2.65 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 3.09 GBytes 2.65 Gbits/sec receiverComparing to vzI replaced vmType to vz and run the same tests: % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with internal VM driver "vz"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/nir/.lima/krunkit/serial*.log")
INFO[0000] [hostagent] [VZ] - vm state change: running
INFO[0006] [hostagent] Started vsock forwarder: 127.0.0.1:51751 -> vsock:22 on VM
INFO[0006] [hostagent] Detected SSH server is listening on the vsock port; changed 127.0.0.1:51751 to proxy for the vsock port
INFO[0007] SSH Local Port: 51751
INFO[0006] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0016] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0017] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0017] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0017] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0017] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0017] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0017] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0017] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell.
% iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51758 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 281 MBytes 2.36 Gbits/sec
[ 5] 1.00-2.00 sec 285 MBytes 2.39 Gbits/sec
[ 5] 2.00-3.00 sec 283 MBytes 2.38 Gbits/sec
[ 5] 3.00-4.00 sec 282 MBytes 2.37 Gbits/sec
[ 5] 4.00-5.00 sec 284 MBytes 2.37 Gbits/sec
[ 5] 5.00-6.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 6.00-7.00 sec 282 MBytes 2.38 Gbits/sec
[ 5] 7.00-8.00 sec 283 MBytes 2.37 Gbits/sec
[ 5] 8.00-9.00 sec 282 MBytes 2.36 Gbits/sec
[ 5] 9.00-10.00 sec 282 MBytes 2.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 2.76 GBytes 2.37 Gbits/sec sender
[ 5] 0.00-9.92 sec 2.76 GBytes 2.39 Gbits/sec receiver
% iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51760 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 549 MBytes 4.61 Gbits/sec
[ 7] 1.00-2.00 sec 543 MBytes 4.55 Gbits/sec
[ 7] 2.00-3.00 sec 543 MBytes 4.56 Gbits/sec
[ 7] 3.00-4.00 sec 548 MBytes 4.59 Gbits/sec
[ 7] 4.00-5.00 sec 546 MBytes 4.58 Gbits/sec
[ 7] 5.00-6.00 sec 549 MBytes 4.60 Gbits/sec
[ 7] 6.00-7.00 sec 543 MBytes 4.56 Gbits/sec
[ 7] 7.00-8.00 sec 540 MBytes 4.53 Gbits/sec
[ 7] 8.00-9.00 sec 554 MBytes 4.65 Gbits/sec
[ 7] 9.00-10.00 sec 548 MBytes 4.60 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 5.33 GBytes 4.58 Gbits/sec sender
[ 7] 0.00-9.94 sec 5.33 GBytes 4.61 Gbits/sec receiverComparing to qemuI changed vmType to qemu and run the same tests. % ./_output/bin/limactl start krunkit
INFO[0000] Using the existing instance "krunkit"
INFO[0000] Starting the instance "krunkit" with internal VM driver "qemu"
INFO[0000] [hostagent] hostagent socket created at /Users/nir/.lima/krunkit/ha.sock
INFO[0000] [hostagent] Using system firmware ("/opt/homebrew/share/qemu/edk2-aarch64-code.fd")
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/nir/.lima/krunkit/serial*.log")
INFO[0001] SSH Local Port: 51767
INFO[0000] [hostagent] Running in plain mode. Mounts, dynamic port forwarding, containerd, etc. will be ignored. Guest agent will not be running. Static port forwarding is allowed.
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0010] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0010] [hostagent] Waiting for the essential requirement 2 of 2: "Explicitly start ssh ControlMaster"
INFO[0010] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0010] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0010] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0010] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0011] READY. Run `ssh -F "/Users/nir/.lima/krunkit/ssh.config" lima-krunkit` to open the shell.
% iperf3 -c 192.168.105.3
Connecting to host 192.168.105.3, port 5201
[ 5] local 192.168.105.1 port 51774 connected to 192.168.105.3 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 360 MBytes 3.02 Gbits/sec
[ 5] 1.00-2.00 sec 364 MBytes 3.06 Gbits/sec
[ 5] 2.00-3.00 sec 438 MBytes 3.67 Gbits/sec
[ 5] 3.00-4.00 sec 367 MBytes 3.08 Gbits/sec
[ 5] 4.00-5.00 sec 467 MBytes 3.92 Gbits/sec
[ 5] 5.00-6.01 sec 473 MBytes 3.95 Gbits/sec
[ 5] 6.01-7.00 sec 470 MBytes 3.96 Gbits/sec
[ 5] 7.00-8.01 sec 428 MBytes 3.57 Gbits/sec
[ 5] 8.01-9.00 sec 364 MBytes 3.07 Gbits/sec
[ 5] 9.00-10.01 sec 426 MBytes 3.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 4.06 GBytes 3.49 Gbits/sec sender
[ 5] 0.00-10.08 sec 4.06 GBytes 3.46 Gbits/sec receiver
% iperf3 -c localhost
Connecting to host localhost, port 5201
[ 7] local 127.0.0.1 port 51777 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate
[ 7] 0.00-1.00 sec 227 MBytes 1.90 Gbits/sec
[ 7] 1.00-2.00 sec 223 MBytes 1.87 Gbits/sec
[ 7] 2.00-3.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 3.00-4.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 4.00-5.00 sec 271 MBytes 2.28 Gbits/sec
[ 7] 5.00-6.00 sec 271 MBytes 2.27 Gbits/sec
[ 7] 6.00-7.00 sec 272 MBytes 2.28 Gbits/sec
[ 7] 7.00-8.00 sec 271 MBytes 2.28 Gbits/sec
[ 7] 8.00-9.00 sec 268 MBytes 2.25 Gbits/sec
[ 7] 9.00-10.00 sec 269 MBytes 2.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 7] 0.00-10.00 sec 2.55 GBytes 2.19 Gbits/sec sender
[ 7] 0.00-10.01 sec 2.55 GBytes 2.19 Gbits/sec receiver |
nirs
reviewed
Oct 14, 2025
|
The default mode shouldn't depend on |
Should I try with gvisor? |
Yes |
|
I think this can work similar to vz:
In both case krunkit will use virtio-net,type=unixstream,mac=,...path=... |
|
Finally networking now works like vz(slirp using gvisor-tap-vsock). Instances can also be run without the
then we are good to go. ➜ lima2 git:(driver/krun) ✗ lm start templates/experimental/krunkit.yaml
? Creating an instance "krunkit" Proceed with the current configuration
INFO[0001] Starting the instance "krunkit" with external VM driver "krunkit"
INFO[0001] Attempting to download the image arch=aarch64 digest="sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8" location="https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
INFO[0001] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/eccac025a7a4709721a97d4ef1d5973546c85062917f188cdc6d247b665cbab5/data"
INFO[0006] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:2f98346ed3dcaf47cfd6461a5685e582284329a1ece1d952ade881b9fe7491e8" location="https://github.com/containerd/nerdctl/releases/download/v2.1.6/nerdctl-full-2.1.6-linux-arm64.tar.gz"
INFO[0006] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/d82077f1308e8d063805c4c1f761e1c3fd636e18cab7e25919a18e7a0838afa2/data"
INFO[0007] [hostagent] hostagent socket created at /Users/ansumansahoo/.lima/krunkit/ha.sock
INFO[0008] SSH Local Port: 65298
INFO[0007] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0017] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0019] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0019] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0029] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0030] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0030] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0030] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0030] [hostagent] Forwarding "/run/lima-guestagent.sock" (guest) to "/Users/ansumansahoo/.lima/krunkit/ga.sock" (host)
INFO[0031] [hostagent] time="2025-10-23T03:17:58+05:30" level=info msg="Getting guest agent connection"
INFO[0030] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0030] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0030] [hostagent] Guest agent is running
INFO[0030] [hostagent] Not forwarding TCP 127.0.0.54:53
INFO[0030] [hostagent] Not forwarding TCP 127.0.0.53:53
INFO[0030] [hostagent] Not forwarding TCP 0.0.0.0:22
INFO[0030] [hostagent] Not forwarding TCP [::]:22
INFO[0030] [hostagent] Not forwarding UDP 127.0.0.54:53
INFO[0030] [hostagent] Not forwarding UDP 127.0.0.53:53
INFO[0030] [hostagent] Not forwarding UDP 192.168.5.15:68
INFO[0043] [hostagent] Forwarding UDP from 0.0.0.0:57561 to 127.0.0.1:57561
INFO[0048] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0048] [hostagent] Waiting for the guest agent to be running
INFO[0048] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0049] [hostagent] Forwarding TCP from 127.0.0.1:36777 to 127.0.0.1:36777
INFO[0060] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0061] READY. Run `limactl shell krunkit` to open the shell. |
afbjorklund
reviewed
Oct 24, 2025
Sure thing It is very easy with minimal changes to the current krunkit branch, we can also support diff --git a/pkg/driver/krunkit/krunkit_darwin.go b/pkg/driver/krunkit/krunkit_darwin.go
index e0288549..ca1cf6fd 100644
--- a/pkg/driver/krunkit/krunkit_darwin.go
+++ b/pkg/driver/krunkit/krunkit_darwin.go
@@ -25,20 +25,22 @@ import (
"github.com/sirupsen/logrus"
)
-const logLevelInfo = "3"
+const logLevelInfo = "info"
// Cmdline constructs the command line arguments for krunkit based on the instance configuration.
func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
var args = []string{
- "--memory", strconv.Itoa(2048),
+ "--memory", strconv.Itoa(6144),
"--cpus", fmt.Sprintf("%d", *inst.Config.CPUs),
"--device", fmt.Sprintf("virtio-serial,logFilePath=%s", filepath.Join(inst.Dir, filenames.SerialLog)),
- "--krun-log-level", logLevelInfo,
+ "--log-level", logLevelInfo,
"--restful-uri", "none://",
+ "--bootloader", fmt.Sprintf("efi,variable-store=%s,create", filepath.Join(inst.Dir, "vfkit-efi")),
// First virtio-blk device is the boot disk
- "--device", fmt.Sprintf("virtio-blk,path=%s,format=raw", filepath.Join(inst.Dir, filenames.DiffDisk)),
+ "--device", fmt.Sprintf("virtio-blk,path=%s", filepath.Join(inst.Dir, filenames.DiffDisk)),
"--device", fmt.Sprintf("virtio-blk,path=%s", filepath.Join(inst.Dir, filenames.CIDataISO)),
+ "--device", "virtio-vsock,port=2222,socketURL=/Users/ansumansahoo/.lima/krunkit/vfkit-2222.sock,connect",
}
networkArgs, err := buildNetworkArgs(inst)
@@ -61,7 +63,7 @@ func Cmdline(inst *limatype.Instance) (*exec.Cmd, error) {
}
args = append(args, networkArgs...)
- cmd := exec.CommandContext(context.Background(), "krunkit", args...)
+ cmd := exec.CommandContext(context.Background(), "vfkit", args...)
return cmd, nil
}
@@ -82,7 +84,7 @@ func buildNetworkArgs(inst *limatype.Instance) ([]string, error) {
return nil, err
}
- args = append(args, "--device", fmt.Sprintf("virtio-net,type=unixgram,fd=%d,mac=%s", client.Fd(), limayaml.MACAddre
ss(inst.Dir)))
+ args = append(args, "--device", fmt.Sprintf("virtio-net,fd=%d,mac=%s", client.Fd(), limayaml.MACAddress(inst.Dir)))
}
for _, nw := range inst.Networks {
diff --git a/pkg/driver/krunkit/krunkit_driver_darwin.go b/pkg/driver/krunkit/krunkit_driver_darwin.go
index 0d42da7b..e449925d 100644
--- a/pkg/driver/krunkit/krunkit_driver_darwin.go
+++ b/pkg/driver/krunkit/krunkit_driver_darwin.go
@@ -26,6 +26,7 @@ import (
type LimaKrunkitDriver struct {
Instance *limatype.Instance
SSHLocalPort int
+ vSockPort int
usernetClient *usernet.Client
stopUsernet context.CancelFunc
@@ -42,6 +43,7 @@ func New() *LimaKrunkitDriver {
func (l *LimaKrunkitDriver) Configure(inst *limatype.Instance) *driver.ConfiguredDriver {
l.Instance = inst
l.SSHLocalPort = inst.SSHLocalPort
+ l.vSockPort = 2222
return &driver.ConfiguredDriver{
Driver: l,
@@ -182,11 +184,13 @@ func (l *LimaKrunkitDriver) Unregister(ctx context.Context) error {
}
func (l *LimaKrunkitDriver) ForwardGuestAgent() bool {
- return true
+ return false
}
func (l *LimaKrunkitDriver) GuestAgentConn(ctx context.Context) (net.Conn, string, error) {
- return nil, "unix", nil
+ var d net.Dialer
+ conn, err := d.DialContext(ctx, "unix", "/Users/ansumansahoo/.lima/krunkit/vfkit-2222.sock")
+ return conn, "", err
}
func (l *LimaKrunkitDriver) Validate(ctx context.Context) error {
@@ -219,6 +223,8 @@ func (l *LimaKrunkitDriver) Info() driver.Info {
if l.Instance != nil && l.Instance.Dir != "" {
info.InstanceDir = l.Instance.Dir
}
+ info.VsockPort = l.vSockPort
+ info.VirtioPort = ""
info.Features = driver.DriverFeatures{
DynamicSSHAddress: false,
➜ lima2 git:(driver/krun) ✗ lm start templates/experimental/krunkit.yaml
? Creating an instance "krunkit" Proceed with the current configuration
INFO[0001] Starting the instance "krunkit" with external VM driver "krunkit"
INFO[0001] Attempting to download the image arch=aarch64 digest="sha256:26d0ac2236f12954923eb35ddfee8fa9fff3eab6111ba84786b98ab3b972c6d8" location="https://cloud-images.ubuntu.com/releases/plucky/release-20250701/ubuntu-25.04-server-cloudimg-arm64.img"
INFO[0001] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/eccac025a7a4709721a97d4ef1d5973546c85062917f188cdc6d247b665cbab5/data"
INFO[0005] Attempting to download the nerdctl archive arch=aarch64 digest="sha256:2f98346ed3dcaf47cfd6461a5685e582284329a1ece1d952ade881b9fe7491e8" location="https://github.com/containerd/nerdctl/releases/download/v2.1.6/nerdctl-full-2.1.6-linux-arm64.tar.gz"
INFO[0005] Using cache "/Users/ansumansahoo/Library/Caches/lima/download/by-url-sha256/d82077f1308e8d063805c4c1f761e1c3fd636e18cab7e25919a18e7a0838afa2/data"
INFO[0006] [hostagent] hostagent socket created at /Users/ansumansahoo/.lima/krunkit/ha.sock
INFO[0006] SSH Local Port: 58583
INFO[0006] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0016] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0019] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0019] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0029] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0029] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0029] [hostagent] Setting up static TCP forwarding from 127.0.0.1:5201 to 127.0.0.1:5201
INFO[0029] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0029] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0029] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0029] [hostagent] time="2025-10-25T20:19:29+05:30" level=info msg="Getting guest agent connection"
INFO[0029] [hostagent] Guest agent is running
INFO[0029] [hostagent] Not forwarding TCP 127.0.0.54:53
INFO[0029] [hostagent] Not forwarding TCP 0.0.0.0:22
INFO[0029] [hostagent] Not forwarding TCP 127.0.0.53:53
INFO[0029] [hostagent] Not forwarding TCP [::]:22
INFO[0029] [hostagent] Not forwarding UDP 127.0.0.54:53
INFO[0029] [hostagent] Not forwarding UDP 127.0.0.53:53
INFO[0029] [hostagent] Not forwarding UDP 192.168.5.15:68
INFO[0042] [hostagent] Forwarding UDP from [::]:57306 to 127.0.0.1:57306
INFO[0044] [hostagent] Forwarding UDP from 0.0.0.0:52876 to 127.0.0.1:52876
INFO[0050] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0050] [hostagent] Waiting for the guest agent to be running
INFO[0050] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0050] [hostagent] Forwarding TCP from 127.0.0.1:37195 to 127.0.0.1:37195
INFO[0062] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0062] READY. Run `limactl shell krunkit` to open the shell. |
|
Let's merge this PR as is (after passing the CI), and implement additional features in follow-up PRs |
|
We would like to merge this and ship a beta release before the meeting on Thursday Is there any blocker other than fixing lint failures? Feel free to ping us if you need any help 💚 |
Thanks for the support, really means a lot! Sorry I took this long but I am very near to completion. Ubuntu 25.04 VM is compatible with Venus and MESA, but not as well as Fedora. While checking GPU info using Devices:
========
GPU0:
apiVersion = 1.2.0
driverVersion = 24.1.5
vendorID = 0x106b
deviceID = 0xf060207
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Virtio-GPU Venus (Apple M1)
driverID = DRIVER_ID_MESA_VENUS
driverName = venus
driverInfo = Mesa 24.1.5
conformanceVersion = 1.3.0.0
deviceUUID = 47063303-a64b-0845-3bf5-01d1daa8649b
driverUUID = 05a3cceb-1991-3628-23bd-764bdc583f20
GPU1:
apiVersion = 1.3.278
driverVersion = 0.0.1
vendorID = 0x10005
deviceID = 0x0000
deviceType = PHYSICAL_DEVICE_TYPE_CPU
deviceName = llvmpipe (LLVM 18.1.6, 128 bits)
driverID = DRIVER_ID_MESA_LLVMPIPE
driverName = llvmpipe
driverInfo = Mesa 24.1.5 (LLVM 18.1.6)
conformanceVersion = 1.3.1.1
deviceUUID = 6d657361-3234-2e31-2e35-000000000000
driverUUID = 6c6c766d-7069-7065-5555-494400000000Now the only problem with Fedora is that it takes significant amount of time to start and thats where I was stuck. After digging deep I found out that many boot scripts are not well compatible with Fedora so will have to hack around with those and make the driver fast. |
unsuman
force-pushed
the
driver/krun
branch
3 times, most recently
from
e7adf1a to
a126fbb
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for the Krunkit VM driver to Lima, enabling super-light VMs on macOS/ARM64 with GPU acceleration via Mesa's Venus Vulkan driver. Krunkit builds on libkrun to provide hardware-isolated VMs with optional GPU support for AI and graphics workloads.
- Implements a complete external driver for Krunkit with VM lifecycle management
- Adds GPU acceleration support for Vulkan workloads via Venus driver on Fedora
- Includes documentation and templates for both general and GPU-enabled configurations
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| website/content/en/docs/config/vmtype/krunkit.md | Documentation for the Krunkit driver with installation instructions and configuration examples |
| templates/experimental/krunkit.yaml | YAML template for GPU-enabled Krunkit VM configuration |
| pkg/driver/krunkit/krunkit_driver_darwin.go | Main driver implementation with VM lifecycle methods and GPU provisioning |
| pkg/driver/krunkit/krunkit_darwin.go | Helper functions for command-line construction, networking, and disk management |
| pkg/driver/krunkit/hack/install-vulkan-gpu.sh | GPU setup script for installing Mesa/Vulkan drivers inside the VM |
| pkg/driver/krunkit/errors_darwin.go | Error definitions for unimplemented driver features |
| cmd/lima-driver-krunkit/main_darwin.go | Entry point for the external Krunkit driver binary |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
pkg/driver/krunkit/krunkit_darwin.go
Outdated
| } | ||
|
|
||
| if len(args) == 0 { | ||
| return args, errors.New("no socket_vment networks defined") |
There was a problem hiding this comment.
Corrected spelling of 'socket_vment' to 'socket_vmnet'.
Suggested change
| return args, errors.New("no socket_vment networks defined") | |
| return args, errors.New("no socket_vmnet networks defined") |
| } | ||
|
|
||
| func (l *LimaKrunkitDriver) BootScripts() (map[string][]byte, error) { | ||
| // Override default reboot-if-required with a no-op because Fedora does not supports this well and |
There was a problem hiding this comment.
Grammatical error: 'does not supports' should be 'does not support'.
Suggested change
| // Override default reboot-if-required with a no-op because Fedora does not supports this well and | |
| // Override default reboot-if-required with a no-op because Fedora does not support this well and |
| cpus: 4 | ||
| arch: aarch64 | ||
|
|
||
| # Fedora 40+ is preferred because of better support for MESA & Vulkan, |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library, as used consistently in the documentation.
Suggested change
| # Fedora 40+ is preferred because of better support for MESA & Vulkan, | |
| # Fedora 40+ is preferred because of better support for Mesa & Vulkan, |
| # Install required packages | ||
| sudo dnf install -y dnf-plugins-core dnf-plugin-versionlock llvm18-libs | ||
|
|
||
| # Install Vulkan and MESA base packages |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
| vulkan-loader \ | ||
| glslc | ||
|
|
||
| # Enable COPR repo with patched MESA for Venus support |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
| # Enable COPR repo with patched MESA for Venus support | ||
| sudo dnf copr enable -y slp/mesa-krunkit fedora-40-aarch64 | ||
|
|
||
| # Downgrade to patched MESA version from COPR |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
| sudo dnf downgrade -y mesa-vulkan-drivers.aarch64 \ | ||
| --repo=copr:copr.fedorainfracloud.org:slp:mesa-krunkit | ||
|
|
||
| # Lock MESA version to prevent automatic upgrades |
There was a problem hiding this comment.
Inconsistent capitalization: 'MESA' should be 'Mesa' to match the proper name of the Mesa 3D Graphics Library.
AkihiroSuda
reviewed
Oct 29, 2025
| Script: ptr.Of(gpuProvisionScript), | ||
| ProvisionData: limatype.ProvisionData{ | ||
| Content: ptr.Of(gpuProvisionScript), | ||
| Path: ptr.Of("/usr/local/bin/install_vulkan_gpu.sh"), |
There was a problem hiding this comment.
The file in the hack directory uses hyphens, not underscores
AkihiroSuda
reviewed
Oct 29, 2025
| return errors.New("krunkit driver requires macOS 13 or higher to run") | ||
| } | ||
| if runtime.GOARCH != "arm64" { | ||
| return errors.New("krunkit driver requires an Apple Silicon (arm64) host") |
There was a problem hiding this comment.
The file should just have _darwin_arm64.go suffix ?
AkihiroSuda
reviewed
Oct 29, 2025
| return fmt.Errorf("unsupported arch: %q (krunkit requires native arch)", *cfg.Arch) | ||
| } | ||
| if _, err := exec.LookPath(vmType); err != nil { | ||
| return errors.New("krunkit CLI not found in PATH. Install it via:\n$ brew tap slp/krunkit\n$ brew install krunkit") |
There was a problem hiding this comment.
maybe drop $ for ease of copypasting
AkihiroSuda
reviewed
Oct 29, 2025
| } | ||
| if krunkitOpts.GPUAccel != nil && *krunkitOpts.GPUAccel { | ||
| if !isFedoraConfigured(cfg) { | ||
| return errors.New("gpuAccel: true requires a Fedora image (use a Fedora base template or image)") |
There was a problem hiding this comment.
probably this should just be a warning
AkihiroSuda
reviewed
Oct 29, 2025
| set -eux -o pipefail | ||
|
|
||
| # Install required packages | ||
| sudo dnf install -y dnf-plugins-core dnf-plugin-versionlock llvm18-libs |
There was a problem hiding this comment.
Suggested change
| sudo dnf install -y dnf-plugins-core dnf-plugin-versionlock llvm18-libs | |
| dnf install -y dnf-plugins-core dnf-plugin-versionlock llvm18-libs |
The entire script should be just executed with sudo
AkihiroSuda
reviewed
Oct 29, 2025
| glslc | ||
|
|
||
| # Enable COPR repo with patched MESA for Venus support | ||
| sudo dnf copr enable -y slp/mesa-krunkit fedora-40-aarch64 |
There was a problem hiding this comment.
I guess for the better compatibility because the developer has made the patch on Fedora 40 and podman also recommends this. See for macOS(Apple Silicon): https://podman-desktop.io/docs/podman/gpu
Containerfile:
FROM fedora:40
USER 0
RUN dnf -y install dnf-plugins-core && \
dnf -y install dnf-plugin-versionlock && \
dnf -y install mesa-vulkan-drivers vulkan-loader-devel vulkan-headers vulkan-tools vulkan-loader glslc && \
dnf -y copr enable slp/mesa-krunkit fedora-40-aarch64 && \
dnf -y downgrade mesa-vulkan-drivers.aarch64 --repo=copr:copr.fedorainfracloud.org:slp:mesa-krunkit && \
dnf versionlock mesa-vulkan-drivers && \
dnf clean all
AkihiroSuda
reviewed
Oct 29, 2025
| vulkan-loader \ | ||
| glslc | ||
|
|
||
| # Enable COPR repo with patched MESA for Venus support |
There was a problem hiding this comment.
AkihiroSuda
reviewed
Oct 29, 2025
|
|
||
| vmOpts: | ||
| krunkit: | ||
| gpuAccel: true |
There was a problem hiding this comment.
Assuming that other drivers will support GPU acceleration too in the future, maybe this can be a proper YAML field and a CLI flag.
Can be discussed in a follow-up issue/PR.
AkihiroSuda
reviewed
Oct 29, 2025
templates/experimental/krunkit.yaml
Outdated
|
|
||
| mounts: | ||
| - location: "~" | ||
| writable: true |
There was a problem hiding this comment.
template://_default/mounts can be used
AkihiroSuda
reviewed
Oct 29, 2025
| | ⚡ Requirement | Lima >= 2.0, macOS >= 13 (Ventura+), Apple Silicon (arm64) | | ||
| | ------------- | ----------------------------------------------------------- | | ||
|
|
||
| Krunkit runs super‑light VMs on macOS/ARM64 with a focus on GPU access. It builds on [libkrun](https://github.com/containers/libkrun?tab=readme-ov-file), a library that embeds a VMM so apps can launch processes in a hardware‑isolated VM (HVF on macOS, KVM on Linux). The standout feature is GPU support in the guest via Mesa’s Venus Vulkan driver ([venus](https://docs.mesa3d.org/drivers/venus.html)), enabling Vulkan workloads inside the VM. See the project: [containers/krunkit](https://github.com/containers/krunkit). |
There was a problem hiding this comment.
Suggested change
| Krunkit runs super‑light VMs on macOS/ARM64 with a focus on GPU access. It builds on [libkrun](https://github.com/containers/libkrun?tab=readme-ov-file), a library that embeds a VMM so apps can launch processes in a hardware‑isolated VM (HVF on macOS, KVM on Linux). The standout feature is GPU support in the guest via Mesa’s Venus Vulkan driver ([venus](https://docs.mesa3d.org/drivers/venus.html)), enabling Vulkan workloads inside the VM. See the project: [containers/krunkit](https://github.com/containers/krunkit). | |
| Krunkit runs super‑light VMs on macOS/ARM64 with a focus on GPU access. It builds on [libkrun](https://github.com/containers/libkrun), a library that embeds a VMM so apps can launch processes in a hardware‑isolated VM (HVF on macOS, KVM on Linux). The standout feature is GPU support in the guest via Mesa’s Venus Vulkan driver ([venus](https://docs.mesa3d.org/drivers/venus.html)), enabling Vulkan workloads inside the VM. See the project: [containers/krunkit](https://github.com/containers/krunkit). |
AkihiroSuda
reviewed
Oct 29, 2025
| ```bash | ||
| brew tap slp/krunkit | ||
| brew install krunkit | ||
| ``` |
There was a problem hiding this comment.
link https://github.com/slp/homebrew-krun for reference
AkihiroSuda
reviewed
Oct 29, 2025
| ``` | ||
|
|
||
| ## Using the driver with Lima (external-first) | ||
| This driver is external-first. Build the driver binary and point Lima to it. See also [Virtual Machine Drivers](../../dev/drivers). |
There was a problem hiding this comment.
"external-first" doesn't make a sense to readers who are not aware of the context.
Maybe just drop it.
AkihiroSuda
reviewed
Oct 29, 2025
| ```bash | ||
| # From the Lima source tree | ||
| go build -o /path/to/bin/lima-driver-krunkit ./cmd/lima-driver-krunkit/main_darwin.go | ||
| export LIMA_DRIVERS_PATH=/path/to/bin |
There was a problem hiding this comment.
Why not use the standard libexec dir?
AkihiroSuda
reviewed
Oct 29, 2025
|
|
||
| ## Notes and caveats | ||
| - macOS Ventura or later on Apple Silicon is required. | ||
| - mountType must be “virtiofs”. |
There was a problem hiding this comment.
reverse-sshfs should work too?
|
Did you get any LLM to work? $ ramalama serve gemma3:1b
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Virtio-GPU Venus (Apple M4 Max) (venus) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 0 | matrix cores: none
Failed to infer a tool call example (possible template bug)
build: 1 (a23b9bd) with cc (GCC) 15.2.1 20250808 (Red Hat 15.2.1-1) for aarch64-redhat-linux
system info: n_threads = 4, n_threads_batch = 4, total_threads = 4
system_info: n_threads = 4 (n_threads_batch = 4) / 4 | CPU : NEON = 1 | ARM_FMA = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
main: binding port with default address family
main: HTTP server is listening, hostname: 0.0.0.0, port: 8080, http threads: 3
main: loading model
srv load_model: loading model '/mnt/models/gemma-3-1b-it-Q4_K_M.gguf'
llama_model_load_from_file_impl: using device Vulkan0 (Virtio-GPU Venus (Apple M4 Max)) (unknown id) - 131072 MiB free
[...]$ ramalama chat "Introduce yourself"
2. The given data is a sample of a larger population. We are interested in estimating the population mean.
a) What is the sample mean?
b) What is the sample standard deviation?
c) What is the estimated population mean?
d) What is the estimated population standard deviation?
[...]Seems to be a bug of something inside krunkit or ramalama though |
Not yet I will do it now! Afaik Edit: But Ramalama uses |
|
I am getting a failed memory allocation error: Logs for running Qwen3-0.6B using llama.cpp
[ansumansahoo@lima-krunkit llama.cpp]$ ./build/bin/llama-cli --version
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Virtio-GPU Venus (Apple M1) (venus) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 0 | matrix cores: none
register_backend: registered backend Vulkan (1 devices)
register_device: registered device Vulkan0 (Virtio-GPU Venus (Apple M1))
register_backend: registered backend CPU (1 devices)
register_device: registered device CPU (CPU)
load_backend: failed to find ggml_backend_init in /home/ansumansahoo.linux/llama.cpp/build/bin/libggml-vulkan.so
load_backend: failed to find ggml_backend_init in /home/ansumansahoo.linux/llama.cpp/build/bin/libggml-cpu.so
version: 6872 (f549b0007)
built with cc (GCC) 15.2.1 20251022 (Red Hat 15.2.1-3) for aarch64-redhat-linux
[ansumansahoo@lima-krunkit llama.cpp]$ export GGML_VK_VISIBLE_DEVICES=0
./build/bin/llama-cli \
-m llms/Qwen3-0.6B-Q4_K_M.gguf \
-p "Write a hello world program in Python:" \
-ngl 28 \
-c 2048 \
-cnv
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Virtio-GPU Venus (Apple M1) (venus) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 0 | matrix cores: none
register_backend: registered backend Vulkan (1 devices)
register_device: registered device Vulkan0 (Virtio-GPU Venus (Apple M1))
register_backend: registered backend CPU (1 devices)
register_device: registered device CPU (CPU)
load_backend: failed to find ggml_backend_init in /home/ansumansahoo.linux/llama.cpp/build/bin/libggml-vulkan.so
load_backend: failed to find ggml_backend_init in /home/ansumansahoo.linux/llama.cpp/build/bin/libggml-cpu.so
build: 6872 (f549b0007) with cc (GCC) 15.2.1 20251022 (Red Hat 15.2.1-3) for aarch64-redhat-linux (debug)
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_load_from_file_impl: using device Vulkan0 (Virtio-GPU Venus (Apple M1)) (unknown id) - 8192 MiB free
llama_model_loader: loaded meta data with 32 key-value pairs and 310 tensors from llms/Qwen3-0.6B-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen3
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Qwen3-0.6B
llama_model_loader: - kv 3: general.basename str = Qwen3-0.6B
llama_model_loader: - kv 4: general.quantized_by str = Unsloth
llama_model_loader: - kv 5: general.size_label str = 0.6B
llama_model_loader: - kv 6: general.repo_url str = https://huggingface.co/unsloth
llama_model_loader: - kv 7: qwen3.block_count u32 = 28
llama_model_loader: - kv 8: qwen3.context_length u32 = 40960
llama_model_loader: - kv 9: qwen3.embedding_length u32 = 1024
llama_model_loader: - kv 10: qwen3.feed_forward_length u32 = 3072
llama_model_loader: - kv 11: qwen3.attention.head_count u32 = 16
llama_model_loader: - kv 12: qwen3.attention.head_count_kv u32 = 8
llama_model_loader: - kv 13: qwen3.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 14: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 15: qwen3.attention.key_length u32 = 128
llama_model_loader: - kv 16: qwen3.attention.value_length u32 = 128
llama_model_loader: - kv 17: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 18: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 19: tokenizer.ggml.tokens arr[str,151936] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 20: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 21: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 22: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 23: tokenizer.ggml.padding_token_id u32 = 151654
llama_model_loader: - kv 24: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 25: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
llama_model_loader: - kv 26: general.quantization_version u32 = 2
llama_model_loader: - kv 27: general.file_type u32 = 15
llama_model_loader: - kv 28: quantize.imatrix.file str = Qwen3-0.6B-GGUF/imatrix_unsloth.dat
llama_model_loader: - kv 29: quantize.imatrix.dataset str = unsloth_calibration_Qwen3-0.6B.txt
llama_model_loader: - kv 30: quantize.imatrix.entries_count u32 = 196
llama_model_loader: - kv 31: quantize.imatrix.chunks_count u32 = 688
llama_model_loader: - type f32: 113 tensors
llama_model_loader: - type q4_K: 168 tensors
llama_model_loader: - type q6_K: 29 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_K - Medium
print_info: file size = 372.65 MiB (5.24 BPW)
load: printing all EOG tokens:
load: - 151643 ('<|endoftext|>')
load: - 151645 ('<|im_end|>')
load: - 151662 ('<|fim_pad|>')
load: - 151663 ('<|repo_name|>')
load: - 151664 ('<|file_sep|>')
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch = qwen3
print_info: vocab_only = 0
print_info: n_ctx_train = 40960
print_info: n_embd = 1024
print_info: n_layer = 28
print_info: n_head = 16
print_info: n_head_kv = 8
print_info: n_rot = 128
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 2
print_info: n_embd_k_gqa = 1024
print_info: n_embd_v_gqa = 1024
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 3072
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: n_expert_groups = 0
print_info: n_group_used = 0
print_info: causal attn = 1
print_info: pooling type = -1
print_info: rope type = 2
print_info: rope scaling = linear
print_info: freq_base_train = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 40960
print_info: rope_finetuned = unknown
print_info: model type = 0.6B
print_info: model params = 596.05 M
print_info: general.name = Qwen3-0.6B
print_info: vocab type = BPE
print_info: n_vocab = 151936
print_info: n_merges = 151387
print_info: BOS token = 11 ','
print_info: EOS token = 151645 '<|im_end|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: PAD token = 151654 '<|vision_pad|>'
print_info: LF token = 198 'Ċ'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: offloading 28 repeating layers to GPU
load_tensors: offloaded 28/29 layers to GPU
load_tensors: CPU_Mapped model buffer size = 121.72 MiB
load_tensors: Vulkan0 model buffer size = 250.93 MiB
.ggml_vulkan: Memory allocation of size 4096 failed.
ggml_vulkan: vk::Device::mapMemory: ErrorMemoryMapFailed
llama_model_load: error loading model: vk::Device::mapMemory: ErrorMemoryMapFailed
llama_model_load_from_file_impl: failed to load model
common_init_from_params: failed to load model 'llms/Qwen3-0.6B-Q4_K_M.gguf', try reducing --n-gpu-layers if you're running out of VRAM
main: error: unable to load modelAnd yes you are correct it's a current limitation of libkrun and ramalama, they use CPU inferencing as of (July 2, 2025) and are actively working to support GPU inferencing. Setting |
|
Oops, I forgot to add support for |
Signed-off-by: Ansuman Sahoo <anshumansahoo500@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for libkrun using their krunkit CLI, a dynamic library that enables programs to run processes in lightweight, isolated environments using KVM virtualization on Linux and HVF on macOS/ARM64. The key feature is to access GPU via Vulkan(Venus) inside the VM.
Implementation Details
Networking: Network connectivity is established using
gvisor-tap-vsockfor slirp functionality. Additional support foruserv2,shared, andbridgednetworking modes is also included.File Sharing:
virtiofsis implemented for high-performance file sharing between the host and the guest VM.Guest-Host Agent Communication: Host-to-guest agent communication currently relies on ssh. There exists a krunkit bug (see
krunkitshould create unix domain socket files containers/krunkit#79) that fails to create the necessary Unix socket file for virtio-vsock communication.GPU Acceleration (Venus) and OS Recommendation
A primary usecase for this integration is to support GPU-accelerated VMs via krunkit, which uses the Venus driver to provide guest-side Vulkan API access. While krunkit can boot the default Ubuntu Lima template, full GPU (Vulkan) acceleration will be unavailable. This is because a specific version of Mesa is required, which is not easily available on Ubuntu.
Fedora 40+ is the recommended as the guest OS for this feature because of patched Mesa Drivers.
Using COPR avoids the complex and error-prone process of manually compiling Mesa from source, which would be required on other systems.
GPU Provisioning
A provisioning script is included to set up the GPU drivers inside the guest VM. This script must be run manually by the user after the VM boots using
sudo install_vulkan_gpu.sh. It is not run automatically during startup as the process adds a significant delay to the boot time.Fedora boot
Lima logs:Venus support inside VM:Ubuntu boot
Lima logs:Venus support inside VM: