Use static libzstd for self-contained builds #87
Conversation
meta-cla
bot
added
the
CLA Signed
There was a problem hiding this comment.
Pull request overview
This PR modifies the build system to statically link libzstd instead of using dynamic linking, making cutracer.so self-contained and portable for environments where the dynamic libzstd may not be available (e.g., Meta's Buck PAR).
Key Changes:
- Implements static libzstd detection in the Makefile with pkg-config and system path fallback
- Updates README installation instructions to recommend static library packages
- Adds
-lpthreaddependency to the linker flags
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| Makefile | Adds logic to dynamically locate and link static libzstd with fallback to dynamic linking; adds -lpthread to linker flags |
| readme.md | Updates installation instructions to mention static library packages for portable builds |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| ```bash | ||
| # Ubuntu/Debian | ||
| sudo apt-get install libzstd-dev |
There was a problem hiding this comment.
The CI setup script still installs libzstd-dev which only includes the dynamic library on Ubuntu/Debian. For consistency with the PR's goal of creating self-contained builds, consider updating this to also install the static library or documenting that CI builds may use dynamic linking.
| 2. Install system dependencies (libzstd static library for self-contained builds): | ||
|
|
||
| ```bash | ||
| # Ubuntu/Debian |
There was a problem hiding this comment.
The installation instructions for Ubuntu/Debian show libzstd-dev, but this package typically only includes the dynamic library. To achieve the self-contained builds mentioned in the PR goal, Ubuntu/Debian users may need additional steps or a different package. Consider clarifying whether libzstd-dev on Ubuntu/Debian includes the static library, or provide explicit instructions for obtaining it (e.g., building from source or a specific package name).
| # Ubuntu/Debian | |
| # Ubuntu/Debian | |
| # On most Ubuntu/Debian systems, libzstd-dev provides both shared and static libs (libzstd.a). | |
| # You can verify this with, e.g.,: dpkg -L libzstd-dev | grep 'libzstd.a' | |
| # If your distribution does not ship the static library in libzstd-dev, you may need to | |
| # build zstd from source or install a distro-specific static libzstd package. |
| $(warning WARNING: libzstd.a not found. Install with: dnf install libzstd-static) | ||
| ZSTD_STATIC := -lzstd | ||
| endif | ||
|
|
There was a problem hiding this comment.
The -lpthread flag has been added here but it's not mentioned in the PR description or changes summary. If this is an unrelated fix or dependency, it should either be documented in the PR description or removed from this PR and addressed separately to keep changes focused.
| # Note: -lpthread is required because the tool/library uses POSIX threads (e.g., std::thread). |
Makefile
Outdated
| $(warning WARNING: libzstd.a not found. Install with: dnf install libzstd-static) | ||
| ZSTD_STATIC := -lzstd |
There was a problem hiding this comment.
When the static library is not found, the code falls back to -lzstd which will use dynamic linking. This means the build will succeed but won't achieve the stated goal of creating a self-contained binary. Consider making this a hard error instead of a warning, or at least make the warning message more explicit that the build will fall back to dynamic linking and won't be portable.
| $(warning WARNING: libzstd.a not found. Install with: dnf install libzstd-static) | |
| ZSTD_STATIC := -lzstd | |
| $(error ERROR: libzstd.a not found. Install with: dnf install libzstd-static. \ | |
| Static libzstd.a is required to produce a self-contained binary; falling back to dynamic libzstd.so is not allowed.) |
- Dynamically search for libzstd.a in common system paths - Improved warning messages when static library not found - explicitly states that build will fall back to dynamic linking and result won't be portable - Added -lpthread to link flags (required by zstd's internal threading) - Updated README with verification steps for Ubuntu/Debian users
FindHao
force-pushed
the
findhao/static-zstd-linking
branch
from
7361543 to
f86d376
Compare
Summary
This PR modifies the build system to statically link libzstd, making cutracer.so self-contained and portable across different environments where the dynamic libzstd may not be available.
Changes
libzstd.ain common system paths (/usr/lib64,/usr/lib,/usr/local/lib64,/usr/local/lib) with pkg-config fallback-lpthreadto link flags (required because zstd uses POSIX threads internally for multi-threaded compression)libzstd.ais availableWhy
When using CUTracer via
CUDA_INJECTION64_PATHin certain environments, the dynamiclibzstd.so.1may not be available in the library search path, causing library load failures. Static linking resolves this dependency issue by embedding zstd into cutracer.so.Test Plan
sudo dnf install libzstd-staticsudo apt install libzstd-dev(verify withdpkg -L libzstd-dev | grep libzstd.a)make clean && makeldd lib/cutracer.so | grep zstd(should be empty)CUDA_INJECTION64_PATHin target environment