Materialize unplugged from global cache#261
Conversation
Adds <globalFolder>/unplugged extracted cache and materializes per-project unplugged dirs from it. macOS uses clonefile(2); Linux uses ioctl(FICLONE) reflinks with a copy fallback. Falls back to zip extraction when cloning/copying isn't possible. Also adds Project::global_unplugged_path and Path::fs_clonefile.
⏱️ Benchmark Resultsgatsby install-full-cold
📊 Raw benchmark data (gatsby install-full-cold)Base times: 2.733s, 2.581s, 2.569s, 2.616s, 2.576s, 2.533s, 2.615s, 2.597s, 2.508s, 2.501s, 2.603s, 2.569s, 2.570s, 2.543s, 2.575s, 2.557s, 2.540s, 2.360s, 2.561s, 2.568s, 2.577s, 2.543s, 2.606s, 2.546s, 2.628s, 2.510s, 2.519s, 2.589s, 2.444s, 2.584s Head times: 2.908s, 2.820s, 2.920s, 2.839s, 2.810s, 2.854s, 2.875s, 2.955s, 2.847s, 2.861s, 2.821s, 2.941s, 2.904s, 2.925s, 2.836s, 2.803s, 2.791s, 2.878s, 2.881s, 2.895s, 2.723s, 2.779s, 2.855s, 2.873s, 2.872s, 2.807s, 2.954s, 2.906s, 2.885s, 2.812s gatsby install-cache-and-lock (warm, with lockfile)
📊 Raw benchmark data (gatsby install-cache-and-lock (warm, with lockfile))Base times: 0.437s, 0.437s, 0.436s, 0.461s, 0.440s, 0.471s, 0.470s, 0.438s, 0.441s, 0.444s, 0.445s, 0.451s, 0.443s, 0.449s, 0.443s, 0.447s, 0.439s, 0.441s, 0.447s, 0.446s, 0.446s, 0.445s, 0.446s, 0.442s, 0.446s, 0.444s, 0.441s, 0.448s, 0.450s, 0.448s Head times: 0.540s, 0.537s, 0.547s, 0.542s, 0.543s, 0.541s, 0.543s, 0.540s, 0.544s, 0.544s, 0.538s, 0.560s, 0.545s, 0.543s, 0.548s, 0.546s, 0.546s, 0.547s, 0.548s, 0.551s, 0.553s, 0.546s, 0.551s, 0.557s, 0.554s, 0.549s, 0.551s, 0.550s, 0.548s, 0.546s |
Gate global unplugged materialization on an existing lockfile to avoid regressing fully-cold installs (no lockfile yet).
Lil' bit worried - if it's slower when there isn't a lockfile, wouldn't it also be slower when there's already one? |
This reverts commit ef706dc.
Based on the current implementation, I don’t see an easy way to avoid touching cold-install performance here—but I might be missing something. If there’s a better approach, I’m very open to it. Otherwise I can follow up by recovering some performance budget elsewhere. |
Adds /unplugged extracted cache and materializes per-project unplugged dirs from it.
macos uses clonefile, Linux uses ioctl reflinks with a copy fallback. Falls back to zip extraction when cloning/copying isn't possible.
On my local macos machine, install-cache-and-lock got :