Packaging: Toolchain for clean builds

[ra-id-j]

Summary

This Ideas thread aims to collect all the discussions concerning packaging of lem using the standard GNU Build System: Autoconf, Automake and Libtool. The discussion aims to incorporate the clean build logic encapsulated in recent flake.nix commits. Ideas from packaging of lem in Guix will also be considered. Contributions from packaging for FreeBSD and OpenBSD are also strongly encouraged.

Background

2025-09-16
Commit be73d0d: Add terminal.so
@jgarte: Hi @Sasanidas @vindarel, what do you think if instead we have a Makefile that would build this binary instead of including a binary directly that we don't know how it was compiled?

2025-09-18
PR #1964: Remove binary files
@mldulaney: Open source software should, by definition, not be distributing binary files. Given the xz backdoor, these increase distrust.

Continuing the discussion below.

[ra-id-j]

Guix build for terminal.c

Reference: comment on build invocation of terminal.c in Guix. Thanks @jgarte.

$COMPILER extensions/terminal/terminal.c -L <lib-dir> -lvterm -Wl,-Bdynamic -o <shared-lib> -shared -fPIC -lutil

Dockerfile build for terminal.c

Comparing the previous to the equivalent build invocation as documented in the Dockerfile:

gcc terminal.c -Wl,-Bstatic -lvterm -Wl,-Bdynamic -o terminal.so -shared -fPIC -lutil

I note that Guix does not compile a static library.

TODO: Decision on static vs dynamic shared library

[ra-id-j]

Contributed lisp build snippet for terminal.so

#1964 (comment)

The path is hardcoded to lib/<OS>/<ARCH>/terminal.so. This is probably best delegated to the configure autoconf script to figure out.

---

;; thanks to April & May on Discord
(let* ((source (asdf:system-relative-pathname :lem-terminal "terminal.c"))
       (lib (asdf:system-relative-pathname
             :lem-terminal (format nil
                                   "~(lib/~A/~A/terminal.so~)"
                                   (uiop:operating-system)
                                   (uiop:architecture)))))
  (unless (probe-file lib)
    (ensure-directories-exist lib)
    #+darwin
    (uiop:run-program (format nil "cc ~A -I/opt/homebrew/include -L/opt/homebrew/lib -lvterm -o ~A -shared -fPIC" 
                              source lib))
    #-darwin
    (uiop:run-program (format nil "cc ~A -lvterm -o ~A -shared -fPIC" source lib))))

=>

cc /home/vindarel/lisp-projects/lem/extensions/terminal/terminal.c -lvterm -o /home/vindarel/lisp-projects/lem/extensions/terminal/lib/linux/x64/terminal.so -shared -fPIC

terminal.c:297:34: error: ‘VTermScreenCellAttrs’ has no member named ‘conceal’
  297 |   return terminal->lastCell.attrs.conceal;
[…]

[ra-id-j]

Nix build framework for lem

Reference: #2019

Important comment by @tani:

Lem when used with Nix relies heavily on Nix's Common Lisp library collection. We have no dependency on qlot.

TODO: List the build steps from flake.nix

[ra-id-j]

Nix Build Script: build-lem.lisp

Reference:

lem/flake.nix

Lines 30 to 48 in 6d259d0

	mkBuildScript =
	{
	entryPoint ? "lem:main",
	}:
	pkgs.writeText "build-lem.lisp" ''
	(defpackage :nix-cl-user (:use :cl))
	(in-package :nix-cl-user)
	;; Load ASDF
	(load "${lem-base.asdfFasl}/asdf.${lem-base.faslExt}")
	(asdf:initialize-output-translations '(:output-translations :disable-cache :inherit-configuration))
	;; Load Systems
	(mapcar #'asdf:load-system (uiop:split-string (uiop:getenv "systems")))
	;; Dump Image
	(setf uiop:*image-entry-point* #'${entryPoint})
	(uiop:dump-image "lem" :executable t :compression t)
	'';

The heavy lifting is all done by asdf and uiop.