Skip to content

gateway start fails on OrbStack (macOS) — K3s containerd cannot mount nested overlayfs #756

@llcopenhagen

Description

@llcopenhagen

Description

openshell gateway start fails on macOS when using OrbStack as the Docker runtime. The K3s agent inside the gateway container cannot start because containerd v2's overlayfs snapshotter fails on OrbStack's VirtioFS filesystem. The agent loops indefinitely and no pods can run.

Environment

  • OpenShell: v0.0.22
  • OS: macOS (Apple Silicon, Darwin 25.1.0)
  • Docker runtime: OrbStack (Docker 28.5.2, Kernel 6.17.8-orbstack)
  • K3s inside gateway: v1.35.3+k3s1
  • containerd inside gateway: v2.2.2-k3s1

Error

The K3s agent logs show this error repeating indefinitely:

"overlayfs" snapshotter cannot be enabled for "/var/lib/rancher/k3s/agent/containerd",
try using "fuse-overlayfs" or "native": failed to mount overlay:
mount source: "overlay", target: "...merged", fstype: overlay, flags: 0,
data: "lowerdir=...lower2:...lower1,upperdir=...upper,workdir=...work",
err: invalid argument

The K3s API server starts (control plane works), but the kubelet/agent never becomes ready. All pods stay in CrashLoopBackOff with exit code 128 and reason StartError: sandbox container "..." is not running.

Root Cause

OrbStack's VirtioFS does not support nested overlayfs mounts inside privileged containers. K3s v1.35.3 ships containerd v2.2.2, which tries the overlayfs snapshotter and fails without falling back to fuse-overlayfs or native.

For comparison, vanilla K3s v1.31.5 (containerd v1.x) works fine on OrbStack because it falls back gracefully.

Steps to Reproduce

# On macOS with OrbStack
openshell gateway start --plaintext

# Gateway says "ready" but no pods actually run.
# Check K3s logs:
docker logs openshell-cluster-openshell 2>&1 | grep "overlayfs"

Suggested Fix

Configure the K3s agent to use fuse-overlayfs or native snapshotter when overlayfs is unavailable. This can be done via:

  • K3s --snapshotter=native flag
  • containerd config template with snapshotter fallback
  • Auto-detection of the host filesystem capabilities during gateway startup

Notes

  • OpenShell already probes for OrbStack's Docker socket (~/.orbstack/run/docker.sock), suggesting OrbStack is an intended runtime target
  • The NemoClaw quickstart docs list "Colima, Docker Desktop" for macOS but not OrbStack explicitly

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions