Skip to content

Batch keyboard builds into groups of 16 per job #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
patcoll merged 6 commits into from
Jan 25, 2026
Merged

Batch keyboard builds into groups of 16 per job #24

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
361ae1d
Batch keyboard builds into groups of 10 per job
patcoll Jan 25, 2026
6c20cff
Upload artifacts even when builds fail
patcoll Jan 25, 2026
957bdf2
Add collect job to merge batch artifacts
patcoll Jan 25, 2026
cf7f394
Upload each firmware file as individual artifact
patcoll Jan 25, 2026
386cc12
Delete firmware artifact after individual uploads
patcoll Jan 25, 2026
b6de242
Increase batch size from 10 to 16 keyboards per job
patcoll Jan 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 89 additions & 7 deletions .github/workflows/keymaps.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,20 @@ jobs:
id: set-matrix
run: |
# Parse keymaps file into JSON array of {keyboard, keymap, artifact_name} objects
# then batch them into groups of 10
keyboards=$(grep -v '^#' keymaps | grep -v '^//' | grep -v '^$' | while read -r line; do
kb=$(echo "$line" | awk '{print $1}')
km=$(echo "$line" | awk '{print $2}')
km="${km:-patcoll}"
artifact_name=$(echo "${kb}-${km}" | tr '/' '_')
echo "{\"keyboard\":\"$kb\",\"keymap\":\"$km\",\"artifact_name\":\"$artifact_name\"}"
done | jq -s -c '.')
done | jq -s -c '
# Split into batches of 16
[range(0; length; 16) as $i | {
batch_id: ($i / 16 | floor),
keyboards: .[$i:$i+16]
}]
')
echo "keyboards=$keyboards" >> "$GITHUB_OUTPUT"

setup:
Expand Down Expand Up @@ -122,16 +129,91 @@ jobs:
cd ${{ github.workspace }}/working_area/qmk_firmware
sudo util/qmk_install.sh -y

- name: Sync keyboard
run: mise run sync "${{ matrix.keyboard }}"

- name: Build ${{ matrix.keyboard }}
run: mise run qmk compile -kb "${{ matrix.keyboard }}" -km "${{ matrix.keymap }}"
- name: Sync and build keyboards
env:
KEYBOARDS_JSON: ${{ toJson(matrix.keyboards) }}
run: |
failed=0
echo "$KEYBOARDS_JSON" | jq -c '.[]' | while read -r item; do
keyboard=$(echo "$item" | jq -r '.keyboard')
keymap=$(echo "$item" | jq -r '.keymap')
echo "::group::Building $keyboard:$keymap"
if ! mise run sync "$keyboard" || ! mise run qmk compile -kb "$keyboard" -km "$keymap"; then
echo "::error::Failed to build $keyboard:$keymap"
echo "1" > /tmp/build_failed
fi
echo "::endgroup::"
done
if [ -f /tmp/build_failed ]; then
exit 1
fi

- uses: actions/upload-artifact@v4
if: always()
with:
name: firmware-${{ matrix.artifact_name }}
name: firmware-batch-${{ matrix.batch_id }}
path: |
${{ github.workspace }}/*.bin
${{ github.workspace }}/*.hex
${{ github.workspace }}/*.uf2

collect:
needs: [build]
if: always()
runs-on: ubuntu-latest
outputs:
files: ${{ steps.list.outputs.files }}
steps:
- uses: actions/download-artifact@v4
with:
pattern: firmware-batch-*
path: firmware
merge-multiple: true

- name: List firmware files
id: list
run: |
files=$(ls firmware/ 2>/dev/null | jq -R -s -c 'split("\n") | map(select(length > 0))')
echo "files=$files" >> "$GITHUB_OUTPUT"
echo "Found files: $files"

- uses: actions/upload-artifact@v4
with:
name: firmware
path: firmware/*

- name: Delete batch artifacts
uses: geekyeggo/delete-artifact@v5
with:
name: firmware-batch-*

upload-individual:
needs: [collect]
if: always() && needs.collect.outputs.files != '[]'
runs-on: ubuntu-latest
strategy:
matrix:
file: ${{ fromJson(needs.collect.outputs.files) }}
steps:
- uses: actions/download-artifact@v4
with:
name: firmware

- name: Get artifact name
id: name
run: echo "name=$(echo '${{ matrix.file }}' | sed 's/\.[^.]*$//')" >> "$GITHUB_OUTPUT"

- uses: actions/upload-artifact@v4
with:
name: ${{ steps.name.outputs.name }}
path: ${{ matrix.file }}

cleanup:
needs: [upload-individual]
if: always()
runs-on: ubuntu-latest
steps:
- name: Delete firmware artifact
uses: geekyeggo/delete-artifact@v5
with:
name: firmware
Loading