Skip to content
Merged

Egl2 #73

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 19 additions & 7 deletions DOCKER.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Docker Guide

The Docker image provides a fully headless rendering environment using
[OSMesa](https://docs.mesa3d.org/osmesa.html) (Mesa's off-screen software
renderer). No display server, `xvfb`, or GPU is required — rendering runs
entirely in software on the CPU.
The Docker image provides a fully headless rendering environment. By default
it uses [OSMesa](https://docs.mesa3d.org/osmesa.html) (Mesa's CPU software
renderer) — no display server, `xvfb`, or GPU required. If a GPU render
device is available, pass `--device /dev/dri/renderD128` to enable EGL GPU
rendering instead; `libegl1` is already included in the image.

The default entry point is `whippersnap4` (four-view batch rendering).
`whippersnap1` (single-view snapshot and rotation video) can be invoked by
Expand Down Expand Up @@ -188,7 +189,18 @@ parent directory to retrieve them on the host.
not root.
- The interactive GUI (`whippersnap`) is **not** available in the Docker image —
it requires a display server and PyQt6, which are not installed.
- Headless rendering uses **OSMesa** (Mesa's CPU software renderer, provided by
the `libosmesa6` system package). No GPU, no `/dev/dri/` device, and no
`--privileged` flag are needed.
- **Default rendering** uses **OSMesa** (Mesa's CPU software renderer, provided
by the `libosmesa6` system package). No GPU or `/dev/dri/` device needed.
- **GPU rendering via EGL** works out of the box — `libegl1` is included in the
image. Pass the render device into the container and WhipperSnapPy will
automatically prefer EGL over OSMesa when `/dev/dri/renderD*` is accessible:
```bash
docker run --rm --init \
--device /dev/dri/renderD128 \
-v /path/to/subject:/subject \
-v /path/to/output:/output \
whippersnappy \
-lh /subject/surf/lh.thickness -rh /subject/surf/rh.thickness \
-sd /subject -o /output/snap4.png
```

7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
FROM python:3.11-slim

# libosmesa6 — OSMesa software renderer for headless OpenGL (no GPU/display needed)
# libgl1 — base OpenGL shared library required by PyOpenGL
# libosmesa6 — OSMesa CPU software renderer (default headless path, no GPU needed)
# libegl1 — EGL dispatch library; enables GPU rendering when /dev/dri/renderD*
# is passed via --device (e.g. docker run --device /dev/dri/renderD128)
# libgl1 — base OpenGL dispatch library required by PyOpenGL
# libglib2.0-0, libfontconfig1, libdbus-1-3 — runtime deps for Pillow / font rendering
RUN apt-get update && apt-get install -y --no-install-recommends \
libosmesa6 \
libegl1 \
libgl1 \
libglib2.0-0 \
libfontconfig1 \
Expand Down
21 changes: 15 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,18 @@ For interactive 3D in Jupyter notebooks:
pip install 'whippersnappy[notebook]'
```

Off-screen (headless) rendering on **Linux** is supported natively via
OSMesa — no `xvfb` or GPU required. On **Windows**, GLFW creates an
invisible window; a GPU driver is sufficient — no display needed. On
**macOS**, a real display connection is required (NSGL does not support
Off-screen (headless) rendering on **Linux** uses a three-path fallback:
1. **GLFW invisible window** — used when a display is available (`DISPLAY` set).
2. **EGL** (GPU, no display needed) — used when no display is detected and a
GPU render device (`/dev/dri/renderD*`) is accessible with `libEGL` installed
(`libegl1` on Debian/Ubuntu). This is the recommended path for SSH servers
with a GPU — no `DISPLAY`, `xvfb`, or OSMesa required.
3. **OSMesa** (CPU software renderer) — final fallback; requires
`sudo apt-get install libosmesa6` (Debian/Ubuntu) or
`sudo dnf install mesa-libOSMesa` (RHEL/Fedora).

On **Windows**, GLFW creates an invisible window; a GPU driver is sufficient.
On **macOS**, a real display connection is required (NSGL does not support
headless rendering).
See the <a href="DOCKER.md">Docker guide</a> for headless Linux usage.

Expand Down Expand Up @@ -209,8 +217,9 @@ See `tutorials/whippersnappy_tutorial.ipynb` for complete notebook examples.

## Docker

The Docker image provides a fully headless OSMesa rendering environment — no
display server, `xvfb`, or GPU required. See <a href="DOCKER.md"><strong>DOCKER.md</strong></a> for details.
The Docker image provides a fully headless rendering environment using
OSMesa (CPU software renderer) — no display server, `xvfb`, or GPU required.
See <a href="DOCKER.md"><strong>DOCKER.md</strong></a> for details.

## API Documentation

Expand Down
11 changes: 7 additions & 4 deletions whippersnappy/gl/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@
This package contains the low-level OpenGL helpers used by the renderers:

- :mod:`~whippersnappy.gl._headless` — headless platform detection; sets
``PYOPENGL_PLATFORM`` on headless Linux *before* any ``OpenGL.GL`` import.
Imported first here and in every GL submodule so the guard takes effect even
when a submodule is imported directly.
``PYOPENGL_PLATFORM`` to ``"egl"`` or ``"osmesa"`` on headless Linux
*before* any ``OpenGL.GL`` import. Imported first here and in every GL
submodule so the guard takes effect even when a submodule is imported
directly. PyOpenGL binds its platform backend on the first
``import OpenGL.GL`` and cannot be changed afterwards, so this ordering
is critical.
- :mod:`~whippersnappy.gl.context` — context lifecycle (create, capture, destroy).
On Linux tries GLFW → EGL (GPU) → OSMesa (CPU) in order.
On Linux tries GLFW → EGL (only when ``PYOPENGL_PLATFORM=egl``) → OSMesa.
- :mod:`~whippersnappy.gl.egl_context` — EGL headless GPU context (Linux).
- :mod:`~whippersnappy.gl.osmesa_context` — OSMesa headless CPU context (Linux).
- :mod:`~whippersnappy.gl.pipeline` — one-time GPU upload: VAO, buffers, uniforms, shader setup.
Expand Down
30 changes: 17 additions & 13 deletions whippersnappy/gl/_headless.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,23 @@

This module MUST be imported before any ``import OpenGL.GL`` statement in the
package. On Linux with no display server it sets ``PYOPENGL_PLATFORM`` so
that PyOpenGL resolves function pointers via the right backend.
that PyOpenGL resolves function pointers via the correct backend before
``OpenGL.GL`` is first imported.

Priority chain when no display is detected (Linux only):

1. **EGL + GPU device** — ``/dev/dri/renderD*`` readable and ``libEGL``
loadable. ``PYOPENGL_PLATFORM`` is left **unset** here so that
:mod:`whippersnappy.gl.egl_context` can set it to ``"egl"`` before
``OpenGL.GL`` is imported.
loadable. Sets ``PYOPENGL_PLATFORM=egl`` immediately so that PyOpenGL
binds function pointers via EGL when ``OpenGL.GL`` is first imported.
2. **OSMesa** — CPU software renderer. Sets ``PYOPENGL_PLATFORM=osmesa``.
3. **Neither** — raises ``RuntimeError`` with install instructions.

When GLFW fails even though ``DISPLAY`` is set (e.g. ``ssh -Y`` with a
forwarded X that lacks GLX 3.3), ``PYOPENGL_PLATFORM`` is already bound to
whatever was set at import time. :func:`init_offscreen_context` handles
this by trying EGL as a second step after GLFW failure, before OSMesa.
When ``DISPLAY`` is set (e.g. normal desktop or ``ssh -Y``), ``_headless``
does not intervene: GLFW is tried first in :func:`init_offscreen_context`.
If GLFW fails (e.g. GLX 3.3 unavailable on the forwarded display), EGL is
attempted only when ``PYOPENGL_PLATFORM`` was already set to ``"egl"`` by
this module at import time — i.e. only for the no-display + EGL-device case.
In all other GLFW-failure scenarios, OSMesa is used as the final fallback.

No OpenGL, GLFW, or other heavy imports are done here — only stdlib.
"""
Expand Down Expand Up @@ -78,11 +80,13 @@ def egl_device_is_available():
and not os.environ.get("WAYLAND_DISPLAY")
):
if egl_device_is_available():
# Defer: egl_context.py will set PYOPENGL_PLATFORM=egl before importing
# OpenGL.GL. Do NOT set osmesa here or GL will bind to the wrong backend.
# Set PYOPENGL_PLATFORM=egl NOW, before OpenGL.GL is imported anywhere.
# PyOpenGL selects its platform backend on first import and cannot be
# changed afterwards; deferring to egl_context.py would mean OpenGL.GL
# is already bound to the wrong backend by the time EGL is tried.
os.environ["PYOPENGL_PLATFORM"] = "egl"
logger.debug(
"No display, EGL + GPU device available — "
"deferring platform selection to EGL context creation."
"No display, EGL + GPU device available — PYOPENGL_PLATFORM=egl set."
)
elif _osmesa_is_available():
os.environ["PYOPENGL_PLATFORM"] = "osmesa"
Expand All @@ -109,6 +113,6 @@ def egl_device_is_available():
elif sys.platform == "linux":
_display = os.environ.get("DISPLAY") or os.environ.get("WAYLAND_DISPLAY")
logger.debug(
"Display set (%s) — will try GLFW; EGL on failure if GPU device available.",
"Display set (%s) — will try GLFW first.",
_display,
)
53 changes: 32 additions & 21 deletions whippersnappy/gl/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,26 @@
Owns the full lifecycle of an OpenGL context — creation, scene rendering,
framebuffer capture, and teardown.

Context creation tries three paths in order (Linux; macOS/Windows use GLFW only):
Context creation tries up to three paths (Linux; macOS/Windows use GLFW only):

1. **GLFW invisible window** — standard path when a display is available.
2. **EGL pbuffer** — headless GPU rendering; no display required.
Tried when GLFW fails for any reason, including when ``DISPLAY`` is set
via ``ssh -Y`` but the forwarded X lacks GLX 3.3. Requires
``libEGL`` and an accessible ``/dev/dri/renderD*`` device node.
3. **OSMesa** — CPU software renderer; no GPU required.
2. **EGL pbuffer** — headless GPU rendering (Linux, no display needed).
Only used when :mod:`~whippersnappy.gl._headless` set
``PYOPENGL_PLATFORM=egl`` at import time (no display + accessible
``/dev/dri/renderD*``). PyOpenGL selects its platform backend on the
first ``import OpenGL.GL`` and cannot be changed afterwards — so EGL is
only safe when it was selected before any ``OpenGL.GL`` import.
3. **OSMesa** — CPU software renderer (Linux only).
Used when neither GLFW nor EGL succeeds.

The :mod:`whippersnappy.gl._headless` guard runs before ``OpenGL.GL`` is
imported and sets ``PYOPENGL_PLATFORM`` appropriately (or defers to EGL).
imported and sets ``PYOPENGL_PLATFORM`` to ``"egl"`` or ``"osmesa"``
as appropriate.
"""
# ruff: noqa: I001 — import order is intentional: _headless must precede OpenGL.GL

import logging
import os
import sys
import warnings
from typing import Any
Expand Down Expand Up @@ -136,15 +140,18 @@ def init_window(width, height, title="WhipperSnapPy", visible=True):
def init_offscreen_context(width, height):
"""Create an invisible OpenGL context for off-screen rendering.

Tries three paths in order on Linux; macOS and Windows use GLFW only.
Tries up to three paths on Linux; macOS and Windows use GLFW only.

1. **GLFW invisible window** — standard path when a display is available.
2. **EGL pbuffer** — headless GPU rendering (Linux only). Attempted
whenever GLFW fails, including when ``DISPLAY`` is set via ``ssh -Y``
but the forwarded X server lacks GLX 3.3. Requires ``libEGL`` and
an accessible ``/dev/dri/renderD*`` device node.
2. **EGL pbuffer** — headless GPU rendering (Linux only, no display needed).
Only attempted when :mod:`~whippersnappy.gl._headless` already set
``PYOPENGL_PLATFORM=egl`` at import time (i.e. no display detected AND
``/dev/dri/renderD*`` is accessible). This guarantees ``OpenGL.GL``
was bound to the EGL backend before any GL call; attempting EGL after
``OpenGL.GL`` has already been imported with a different backend would
silently break function resolution.
3. **OSMesa** — CPU software renderer (Linux only). Used when neither
GLFW nor EGL succeeds.
GLFW nor EGL succeeds, or when ``PYOPENGL_PLATFORM=osmesa`` was set.

Parameters
----------
Expand Down Expand Up @@ -181,10 +188,13 @@ def init_offscreen_context(width, height):
)

# --- Step 2: EGL headless GPU rendering ---
# Tried whenever GLFW fails: no display, or DISPLAY set but GLX unavailable
# (e.g. ssh -Y with software X forwarding that lacks GLX 3.3).
from ._headless import egl_device_is_available # noqa: PLC0415
if egl_device_is_available():
# Only safe when PYOPENGL_PLATFORM=egl was set by _headless.py before
# OpenGL.GL was imported — meaning the process has no display AND an EGL
# device was found at import time. PyOpenGL binds its platform backend on
# first import and cannot be switched afterwards; importing egl_context.py
# here when PYOPENGL_PLATFORM is already something else (e.g. "osmesa" or
# unset/GLX) would cause silent function-pointer mismatches.
if os.environ.get("PYOPENGL_PLATFORM") == "egl":
logger.info("GLFW failed — trying EGL headless GPU rendering.")
try:
from .egl_context import EGLContext # noqa: PLC0415
Expand Down Expand Up @@ -218,14 +228,14 @@ def terminate_context(window):
"""Release the active OpenGL context regardless of how it was created.

This is a drop-in replacement for ``glfw.terminate()`` that also
handles the OSMesa headless path. Call it at the end of every rendering
function instead of calling ``glfw.terminate()`` directly.
handles EGL and OSMesa headless paths. Call it at the end of every
rendering function instead of calling ``glfw.terminate()`` directly.

Parameters
----------
window : GLFWwindow or None
The GLFW window handle returned by :func:`init_offscreen_context`,
or ``None`` when an OSMesa context is active.
or ``None`` when an EGL or OSMesa offscreen context is active.
"""
global _offscreen_context
if _offscreen_context is not None:
Expand All @@ -249,7 +259,8 @@ def capture_window(window):
Parameters
----------
window : GLFWwindow or None
GLFW window handle, or ``None`` when an OSMesa context is active.
GLFW window handle, or ``None`` when an EGL or OSMesa offscreen
context is active.

Returns
-------
Expand Down
24 changes: 17 additions & 7 deletions whippersnappy/gl/egl_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,24 @@
if sys.platform == "darwin":
raise ImportError("EGL is not available on macOS; use GLFW/CGL instead.")

# Must be set before OpenGL.GL is imported anywhere in the process.
# If already set (e.g. user set it, or GLFW succeeded), respect it.
# We set it here because this module is only imported when EGL is needed.
if os.environ.get("PYOPENGL_PLATFORM") != "egl":
# PYOPENGL_PLATFORM=egl should already be set by _headless.py before
# OpenGL.GL was first imported in this process. This guard is a safety net
# for the case where egl_context is imported in isolation (e.g. in tests).
_current_platform = os.environ.get("PYOPENGL_PLATFORM")
if _current_platform is None:
os.environ["PYOPENGL_PLATFORM"] = "egl"

import OpenGL.GL as gl
from PIL import Image
elif _current_platform != "egl":
raise RuntimeError(
f"PYOPENGL_PLATFORM is already set to {_current_platform!r}, "
"but whippersnappy.gl.egl_context requires 'egl'. "
"Importing egl_context with a different platform already bound would "
"cause silent GL function-pointer mismatches. "
"Unset PYOPENGL_PLATFORM or set it to 'egl' before importing any "
"OpenGL modules."
)

import OpenGL.GL as gl # noqa: E402
from PIL import Image # noqa: E402

logger = logging.getLogger(__name__)

Expand Down