Update problemtools packaging #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

gkreitz merged 11 commits into Kattis:develop from gkreitz:297_modernize_build

Apr 24, 2025

.github/workflows/python-app.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -13,8 +13,7 @@ permissions: @@
       contents: read
     jobs:
-      build:
+      pythontests:
         runs-on: ubuntu-latest
         strategy:
           matrix:
@@ Expand All / @@ -25,9 +24,6 @@ jobs: @@
           uses: actions/setup-python@v5
           with:
             python-version: ${{ matrix.python-version }}
-        - name: Install apt packages (for debbuild)
-          run: sudo apt-get install debhelper dh-python python3-pytest libboost-regex-dev build-essential
-          shell: bash
         - name: Install dependencies
           run: |
             python -m pip install --upgrade pip
@@ Expand All / @@ -40,10 +36,18 @@ jobs: @@
             flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics --exclude examples/
             # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
             flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics --exclude examples/
-        - name: Build debian packages
-          run:  make builddeb
         - name: Test with pytest
           run:  pytest
         - name: Run mypy
           run: |
             mypy --non-interactive --config-file mypy.ini -p problemtools
+      packages: # Use a separate job to test debian packaging to speed things up (no need to test this for every python version above)
+        runs-on: ubuntu-latest
+        steps:
+        - uses: actions/checkout@v4
+        - name: Install apt packages (for debbuild)
+          run: sudo apt-get install debhelper dh-virtualenv dpkg-dev python3-venv automake g++ make libboost-regex-dev libgmp-dev python3 git build-essential
+          shell: bash
+        - name: Build debian packages
+          run:  make builddeb

.gitignore

-Original file line number
+Diff line change
@@ Expand Up / @@ -6,6 +6,7 @@ @@
     /support/default_validator/default_validator
     /support/interactive/interactive
     build/
+    /problemtools/_version.py
     venv/
     .pytest_cache/
@@ Expand Down @@

Makefile

-Original file line number
+Diff line change
@@ Expand Up / @@ -8,3 +8,7 @@ checktestdata: support/checktestdata/bootstrap @@
     support/checktestdata/bootstrap:
     	git submodule update --init
+    clean:
+    	make -C support clean
+    	rm -rf problemtools.egg-info build

README.md

-Original file line number
+Diff line change
@@ Expand Up @@
     There are four supported ways of installing and running problemtools.
     (For non-Linux users, "Method 2" below, to use Docker, is probably the least painful.)
-    ### Method 1: Install the Python package
+    ### Method 1: Install the Python package using pipx
     Run
     ```
-    sudo pip3 install git+https://github.com/kattis/problemtools
+    pipx install git+https://github.com/kattis/problemtools
     ```
-    Or if you don't want a system-wide installation,
-    ```
-    pip3 install --user git+https://github.com/kattis/problemtools
-    ```
-    With this second option, in order to get the command line scripts, you need
-    to make sure that the local user bin path used (e.g., on Linux,
-    `$HOME/.local/bin`) is in your `$PATH`.
+    In order to get the command line scripts, you need to make sure that the local
+    user bin path used (e.g., on Linux, `$HOME/.local/bin`) is in your `$PATH`. See
+    [pipx' installation instructions](https://pipx.pypa.io/stable/installation/)
+    for information on how to install `pipx` and set up your `$PATH`.
     In order for problemtools to build and run properly, you also need to have LaTeX
     and various LaTeX packages installed.  See [Requirements and
@@ Expand Down Expand Up @@
     kept up to date).
-    ### Method 3: Run directly from the repository.
+    ### Method 3: Run directly from the repository
-    If you intend to help develop problemtools, or if you just want a
-    bare-bones way of running them, this is your option.
+    If you intend to help develop problemtools, or if you just want a bare-bones
+    way of running them, this is your option.
     For this method, you need to clone the repository (just downloading a
     zip archive of it does not work because the project has submodules
     that are not included in that zip archive).
+    Start by setting up your venv, e.g.,
+        python3 -m venv venv
+        venv/bin/pip install -r requirements.txt
     In order for the tools to work, you first have to compile the various
     support programs, which can be done by running `make` in the root
     directory of problemtools.
@@ Expand Down Expand Up / @@ -134,7 +136,7 @@ root of the repository). @@
     Apart from the build dependencies listed [below](#ubuntu), building
     the Debian package requires that the following tools are installed:
-        debhelper dh-python dpkg-dev
+        debhelper dh-virtualenv dpkg-dev
     The package can then be installed using (replace `<version>` as appropriate):
@@ Expand Down Expand Up / @@ -203,11 +205,11 @@ and a LaTeX installation. @@
     The dependencies needed to *build/install* problemtools can be installed with:
-        sudo apt install automake g++ make libboost-regex-dev libgmp-dev libgmp10 libgmpxx4ldbl python3 python3-pytest python3-setuptools python3-yaml python3-plastex
+        sudo apt install python3-venv automake g++ make libboost-regex-dev libgmp-dev python3 git
     And the dependencies needed to *run* problemtools can be installed with:
-        sudo apt install ghostscript libgmpxx4ldbl python3-minimal python-pkg-resources python3-plastex python3-yaml texlive-fonts-recommended texlive-lang-cyrillic texlive-latex-extra texlive-plain-generic tidy
+        sudo apt install ghostscript python3 texlive-fonts-recommended texlive-lang-cyrillic texlive-latex-extra texlive-plain-generic tidy dvisvgm
     ### Fedora
@@ Expand Down @@

bin/.run_in_venv.sh

-Original file line number
+Diff line change
@@ -0,0 +1,19 @@
+    #!/usr/bin/env bash
+    #
+    # Helper script for the other wrapper scripts to check that a venv exists, or
+    # give a helpful message if it doesn't.
+    VENVPATH="$(dirname "$(dirname "$(readlink -f "$0")")")/venv"
+    if [ ! -x "$VENVPATH/bin/python" ]; then
+        echo "I could not find a python venv at $VENVPATH."
+        echo "To use these wrapper scripts, please set up a venv by:"
+        echo " cd $(dirname "$VENVPATH")"
+        echo " python3 -m venv venv"
+        echo " venv/bin/pip install -r requirements.txt"
+        exit 1
+    fi
+    export PYTHONPATH
+    PYTHONPATH="$(dirname "$(dirname "$(readlink -f "$0")")")${PYTHONPATH:+:}$PYTHONPATH"
+    exec "$VENVPATH/bin/python" -m "$@"

bin/problem2html.sh

-Original file line number
+Diff line change
@@ Expand Up / @@ -5,6 +5,4 @@ @@
     # installing problemtools on the system properly, this script should
     # not be used.
-    export PYTHONPATH
-    PYTHONPATH="$(dirname "$(dirname "$(readlink -f "$0")")")${PYTHONPATH:+:}$PYTHONPATH"
-    exec python3 -m problemtools.problem2html "$@"
+    exec "$(dirname "$(readlink -f "$0")")/.run_in_venv.sh" problemtools.problem2html "$@"

bin/problem2pdf.sh

-Original file line number
+Diff line change
@@ Expand Up / @@ -5,6 +5,4 @@ @@
     # installing problemtools on the system properly, this script should
     # not be used.
-    export PYTHONPATH
-    PYTHONPATH="$(dirname "$(dirname "$(readlink -f "$0")")")${PYTHONPATH:+:}$PYTHONPATH"
-    exec python3 -m problemtools.problem2pdf "$@"
+    exec "$(dirname "$(readlink -f "$0")")/.run_in_venv.sh" problemtools.problem2pdf "$@"

bin/verifyproblem.sh

-Original file line number
+Diff line change
@@ Expand Up / @@ -5,6 +5,4 @@ @@
     # installing problemtools on the system properly, this script should
     # not be used.
-    export PYTHONPATH
-    PYTHONPATH="$(dirname "$(dirname "$(readlink -f "$0")")")${PYTHONPATH:+:}$PYTHONPATH"
-    exec python3 -m problemtools.verifyproblem "$@"
+    exec "$(dirname "$(readlink -f "$0")")/.run_in_venv.sh" problemtools.verifyproblem "$@"

debian/compat

This file was deleted.

debian/control

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -2,13 +2,13 @@ Source: kattis-problemtools
  
    Section: devel

    Priority: optional

    Maintainer: Per Austrin <austrin@kattis.com>

    Build-Depends: debhelper (>= 8.0.0), g++ (>= 4.8), dh-python, python3, python3-setuptools, python3-pytest, python3-yaml, python3-setuptools, python3-pytest, libboost-regex-dev, libgmp-dev, automake, autoconf

    Build-Depends: debhelper-compat (= 13), g++ (>= 4.8), dh-virtualenv, python3, libboost-regex-dev, libgmp-dev, automake, autoconf, git

    Standards-Version: 3.9.4

    Homepage: https://github.com/Kattis/problemtools

    Package: kattis-problemtools

    Architecture: any

    Depends: ${shlibs:Depends}, ${python3:Depends}, ${misc:Depends}, python3-plastex, python3-pkg-resources, texlive-plain-generic, texlive-fonts-recommended, texlive-latex-extra, texlive-lang-cyrillic, tidy, ghostscript, dvisvgm

    Depends: ${shlibs:Depends}, ${misc:Depends}, python3, texlive-plain-generic, texlive-fonts-recommended, texlive-latex-extra, texlive-lang-cyrillic, tidy, ghostscript, dvisvgm

    Recommends: gcc, g++

    Description: Kattis Problem Tools

     These are tools to manage and verify problem packages in the

debian/kattis-problemtools.links

-Original file line number
+Diff line change
@@ -0,0 +1,3 @@
+    opt/venvs/kattis-problemtools/bin/verifyproblem usr/bin/verifyproblem
+    opt/venvs/kattis-problemtools/bin/problem2pdf usr/bin/problem2pdf
+    opt/venvs/kattis-problemtools/bin/problem2html usr/bin/problem2html

debian/rules

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -4,12 +4,17 @@
  
    # Uncomment this to turn on verbose mode.

    #export DH_VERBOSE=1

    # Uncomment this to turn off cleanup.

    export PYBUILD_DISABLE=clean

    %:

    	dh $@ --with python-virtualenv

    export PYBUILD_AFTER_CLEAN=make -C support distclean

    export PYBUILD_TEST_PYTEST=1

    export no_proxy=github.com

    override_dh_virtualenv:

    	dh_virtualenv --builtin-venv --python /usr/bin/python3

    %:

    	dh $@ --with python3 --buildsystem=pybuild

    override_dh_strip:

    	dh_strip --exclude=/PIL/ --exclude=/pillow.libs/

    override_dh_shlibdeps:

    	dh_shlibdeps -X/x86/ -X/PIL/.libs/ -X/pillow.libs/

    override_dh_dwz:

    	dh_dwz --exclude=/PIL/ --exclude=/pillow.libs/

problemtools/__init__.py

Original file line number	Diff line number	Diff line change
		@@ -1 +0,0 @@
		from ._version import __version__

problemtools/_version.py

This file was deleted.

pyproject.toml

-Original file line number
+Diff line change
@@ -0,0 +1,34 @@
+    [build-system]
+    requires = ["setuptools >= 77.0.0", "setuptools-scm"]
+    build-backend = "setuptools.build_meta"
+    [project]
+    name = "problemtools"
+    authors = [
+        {name = "Per Austrin", email = "austrin@kattis.com"},
+    ]
+    description = "Kattis Problem Tools"
+    readme = "README.md"
+    license = "MIT"
+    dependencies = [
+        "PyYAML",
+        "plasTeX>=3.0",
+    ]
+    dynamic = [ "version" ]
+    [project.scripts]
+    verifyproblem = "problemtools.verifyproblem:main"
+    problem2html = "problemtools.problem2html:main"
+    problem2pdf = "problemtools.problem2pdf:main"
+    [project.urls]
+    Repository = "https://github.com/Kattis/problemtools"
+    [tool.setuptools]
+    include-package-data = true
+    [tool.setuptools.packages.find]
+    include = ["problemtools", "problemtools.*"]
+    [tool.setuptools_scm]
+    version_file = "problemtools/_version.py"

requirements.txt

-Original file line number
+Diff line change
@@ -1 +1,2 @@
-    -e .
+    PyYAML
+    plasTeX>=3.0

setup.cfg

This file was deleted.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update problemtools packaging #298

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!