feat(ci): migrate to gitleaks and optimize docs deployments #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cameronraysmith
temporarily deployed
to
preview
GitHub Actions
Inactive
Add gitleaks package to devshell and configure pre-commit hook to scan staged changes for hardcoded secrets. This provides defense-in-depth against accidental secret commits using open-source tooling without external API dependencies. - Add gitleaks to secrets management packages - Configure gitleaks pre-commit hook via git-hooks.nix - Hook runs on staged files with --redact flag to protect secret values
Add justfile recipes for running gitleaks secret scanning both on the entire repository and on staged changes. These recipes provide convenient local secret scanning capabilities. - scan-secrets: scan entire repository for secrets - scan-staged: scan only staged changes (matches pre-commit hook)
Add gitleaks-based secrets scanning CI job to replace GitGuardian. This uses open-source tooling without external API dependencies while maintaining comprehensive secret detection capabilities. - Replace scan job with secrets-scan job - Use gitleaks via justfile recipe - Update job dependency in set-variables
Remove GITGUARDIAN_API_KEY from ghsecrets recipe and check-secrets pattern since we've migrated to open-source gitleaks for secret scanning.
Optimize Cloudflare Workers deployment to use tested preview versions instead of rebuilding. This follows Cloudflare's recommended versions and deployments pattern for safer, faster production deployments. Key improvements: - Tag versions with 12-char git SHA for version tracking - Promote tested preview versions to production (no rebuild) - Add deployment messages for audit trail (local vs CI) - Support graceful fallback if version not found Updated recipes: - cf-deploy-preview: upload with SHA tag, message, and preview alias - cf-deploy-production: query for existing version, promote or build+deploy
Update deploy-docs workflow to use the new justfile recipes that implement version promotion. This enables safer deployments by promoting tested preview versions instead of rebuilding. Changes: - Replace direct wrangler calls with justfile recipes - Add GITHUB_ACTIONS, GITHUB_ACTOR, GITHUB_WORKFLOW env vars - Preview: use just cf-deploy-preview - Production: use just cf-deploy-production (auto-promotes if available)
Add gitleaks:allow comments to suppress false positives on age public keys. Age public keys are safe to commit (they're encryption keys, not secrets), but gitleaks flags them as generic-api-key due to base64-like format. Files updated: - ci-cd-setup.md: CI age key documentation (age public key) - sops-bootstrap.sh: placeholder age keys for bootstrap detection
Add .gitleaksignore file containing fingerprints of known false positives (age public keys in documentation and scripts). This follows gitleaks community best practices for CI environments. Key advantages over baseline files: - Safe to commit (only fingerprints, no secrets) - Stable (no --redact flag matching issues) - Platform-agnostic (no path separator issues) - Simple (one fingerprint per line with comments) All 6 findings are age public keys used as: - Documentation examples in CI/CD setup guides - Placeholder values for sops key rotation detection - Example keys in bootstrap scripts Reference: gitleaks/gitleaks#1263
Add support for triggering preview deployments by adding the
'docs-preview' label to pull requests. This enables on-demand testing
of documentation changes before merging.
When the label is present:
- DEPLOY_ENABLED is set to "true"
- DEPLOY_ENVIRONMENT is set to "preview"
- deploy job runs and calls cf-deploy-preview recipe
- Version is uploaded with branch-specific preview alias
This preserves the sophisticated version promotion workflow while
adding label-based deployment control, following a pattern similar
to existing skip-ci and actions-debug labels.
Preview URL format: https://b-{branch}-ts-nix-docs.sciexp.workers.dev
Add preview-version.sh script from infra to preview semantic-release versions. This script simulates merging the current branch into a target branch and runs semantic-release in dry-run mode to show what version would be released. The script: - Creates temporary git worktree and merge commit for analysis - Restores original branch state via cleanup function - Supports both root and package-level version previews - Uses bun for fast dependency installation via global cache
Add three new recipes to CI/CD group for dynamic package discovery: - list-packages: Lists all package names in packages/ directory - list-packages-json: Outputs packages in JSON format for CI matrix usage - validate-package: Checks that a package has valid structure These recipes enable automated package discovery for CI workflows and monorepo tooling without hardcoding package names.
Add preview-version recipe to justfile in Release group. This recipe allows previewing what version semantic-release would produce if the current branch were merged to a target branch. The recipe delegates to scripts/preview-version.sh with proper parameter handling for both root and package-level version previews.
Replace custom setup-nix action with standardized version from infra: - Use nix-quick-install-action for all installs (removes DeterminateSystems dependency) - Add nix-community/cache-nix-action for GitHub Actions caching on Linux - Change hatchet-protocol from 'rampage' to 'carve' (less aggressive) - Add pinned SHA commits for all actions (security best practice) - Standardize input parameters and add cache outputs - Simplify cachix integration with cachix-action instead of sops/shell scripts
Add reusable GitHub Actions workflow for semantic-release of TypeScript packages: - Based on python-nix-template pattern, adapted for TypeScript/bun - Uses cycjimmy/semantic-release-action@v4 with semantic-release v24 - Supports workflow_dispatch and workflow_call triggers - Supports dry-run mode for testing releases - Uses bun instead of uv/yarn for package management - Removed Python-specific steps (uv build, PyPI publish) - Outputs: version, released, tag for downstream workflows
Add just recipe to release specific packages with semantic-release: - Supports dry_run mode for testing (defaults to false) - Navigates to package directory and runs bun semantic-release - Provides warning message before real releases - Part of Release group
Replace inline test job with package-test.yaml workflow_call and add release orchestration jobs. Key changes: - Add dynamic package discovery in set-variables job using just - Replace hardcoded test matrix with fromJson(packages) output - Add test-release-packages job (PR dry-run validation) - Add release-packages job (main/beta actual releases) - Update deploy job to depend on release-packages Preserves all existing conditional logic, concurrency controls, and workflow_dispatch job selection. Matrix now dynamically discovers packages instead of hardcoded list.
Remove git plugin per semantic-release continuous release best practices. GitHub Releases are generated by @semantic-release/github without needing CHANGELOG.md commits that complicate truly continuous releases.
cameronraysmith
force-pushed
the
gitleaks
branch
from
143ddd0 to
72ec2b5
Compare
GitHub Actions does not support concurrency blocks at the caller level when using reusable workflows (uses:). Removed from test, test-release-packages, and release-packages jobs.
GITHUB_TOKEN is automatically inherited by reusable workflows and cannot be explicitly declared in workflow_call secrets. Removed to fix workflow validation.
Change astro.config.ts to use prerender: true for static site generation at build time, matching the infra repository pattern. This improves performance by serving pre-generated HTML instead of SSR on every request.
Add beta branch to semantic-release configuration to enable prerelease version testing before promoting to main. This matches the infra repository pattern and allows safer release testing.
Apply all 10 commits of fixes from infra repository (5a01715..1b7048c): - Add remote-tracking branch restoration (0cd325c, cb59f3a) Save and restore origin/main to prevent sync issues with semantic-release - Wrap commands with nix develop (infra pattern) Ensures consistent execution environment for bun and semantic-release - Add GITHUB_REF override (f0a1c6c) Fixes semantic-release branch detection in preview context - Sync remote-tracking branch with temp commit (0cd325c) Ensures semantic-release sees local and remote branches as synchronized These fixes resolve semantic-release preview failures and ensure correct version detection in PR context.
Major architectural change to match infra pattern: REMOVED: - cycjimmy/semantic-release-action (black box abstraction) - oven-sh/setup-bun (redundant with Nix environment) - Action-based output extraction ADDED: - Direct semantic-release execution via justfile - Nix environment for consistent execution - fetch-tags: true for reliable version detection - Git-based output extraction (more reliable) - setup-cachix: true for better caching BENEFITS: - Full control over semantic-release execution - Can apply all infra fixes (GITHUB_REF, git identity, etc.) - Consistent with development environment - More transparent execution (no black box) - Easier debugging and customization This enables all subsequent infra improvements to be applied.
Major architectural transformation matching infra pattern: JOB CHANGES: 1. test-release-packages → preview-release-version (inline, PR-only) - NO dependency on test (fast feedback) - Inline steps (transparent, debuggable) - All infra fixes applied (PR branch checkout, git identity, etc.) 2. NEW: preview-docs-deploy (PR-only) - Automatic deployment on all PRs - NO dependency on test (fast feedback) - Parallel to preview-release-version 3. release-packages → production-release-packages - Clear production vs preview distinction - Added SOPS_AGE_KEY secret passing - Maintains quality gates (test+nix required) 4. deploy → production-docs-deploy - Updated dependency: production-release-packages - Removed nix dependency (not required for docs) - Clear production naming CRITICAL PATH IMPROVEMENT: Before: secrets-scan → set-variables → test → test-release-packages (5-10 min) After: secrets-scan → set-variables → preview-release-version (<2 min) BENEFITS: - Fast PR feedback (no test blocking) - Clear preview vs production separation - Transparent execution (inline jobs) - All infra fixes enabled - Automatic preview deployments
Simplify docs deployment by matching infra pattern: REMOVED: - Separate build and deploy jobs - Artifact upload/download mechanism - Conditional build checks - Redundant Nix setup steps CONSOLIDATED TO SINGLE JOB: - deploy-docs job does both build and deploy - Simpler execution flow - No artifact serialization overhead - Fewer moving parts to debug BENEFITS: - Faster execution (no artifact overhead) - Simpler architecture (easier to maintain) - More reliable (no artifact expiration issues) - Consistent with infra pattern - Better resource utilization The justfile recipes (cf-deploy-preview, cf-deploy-production) handle the build internally, so separate build job was unnecessary duplication.
Add FAST_FORWARD_PAT to the list of secrets uploaded to GitHub by the ghsecrets recipe. This token is required for the PR fast-forward merge workflow (/fast-forward command). The token is stored in vars/shared.yaml (SOPS-encrypted) and can be rotated using: just edit-secrets, just ghsecrets, then revoke old token.
Replace nixpkgs playwright-driver with playwright-web-flake's versioned driver to ensure browser versions exactly match @playwright/test 1.56.1. Changes: - Add playwright-web-flake input to flake.nix (pinned to 1.56.1) - Update devshell.nix to use versioned playwright driver - Set PLAYWRIGHT_BROWSERS_PATH to /nix/store path from flake - Add PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 to prevent npm downloads - Add system parameter to perSystem for accessing flake packages This ensures consistent browser versions across all environments and prevents runtime version mismatches between @playwright/test and browsers.
Rename and expand docs recipes to support multi-package monorepo: Renamed recipes (with pattern change): - dev → docs-dev - build → docs-build - preview → docs-preview - linkcheck → docs-linkcheck - optimize-favicon → docs-optimize-favicon Added missing recipes: - docs-format: Format documentation code - docs-lint: Lint documentation code - docs-check: Check and fix documentation code - docs-test: Run all documentation tests - docs-test-unit: Run documentation unit tests - docs-test-e2e: Run documentation E2E tests - docs-test-coverage: Generate test coverage report Pattern change: - FROM: bun run --filter '@typescript-nix-template/docs' [command] - TO: cd packages/docs && bun run [command] This enables users to add additional packages to ./packages/ without recipe name collisions. The cd pattern is more explicit and scalable. Matches proven pattern from infra/ repository for monorepo support.
Update recipe calls to match renamed justfile recipes: - test-coverage → docs-test-coverage - build → docs-build - test-e2e → docs-test-e2e This maintains consistency with the multi-package monorepo pattern where all docs-specific recipes have the docs- prefix.
Change linkcheck → docs-linkcheck to match renamed justfile recipe. Maintains consistency with multi-package monorepo naming pattern.
Update all flake inputs: - Added playwright-web-flake input (pinned to 1.56.1) - Updated nixpkgs (2025-10-02 → 2025-10-28) - Updated flake-parts (2025-10-01 → 2025-10-20) - Updated git-hooks (2025-10-03 → 2025-10-17) The playwright-web-flake input provides version-locked browsers that exactly match the @playwright/test version in package.json.
Add missing just.tmLanguage.json grammar file to enable Just syntax highlighting in documentation code blocks. Changes: - Create packages/docs/src/grammars/ directory - Copy just.tmLanguage.json from infra repository - Fix astro.config.ts to use starlight's expressiveCode.shiki.langs instead of top-level markdown.shikiConfig.langs Starlight uses Expressive Code for syntax highlighting, which requires the grammar to be configured under starlight.expressiveCode.shiki.langs.
Change code block language identifier from 'justfile' to 'just' to match the registered grammar name in just.tmLanguage.json. This resolves the warning about language not being found during build.
cameronraysmith
had a problem deploying
to
preview
GitHub Actions
Failure
cameronraysmith
temporarily deployed
to
preview
GitHub Actions
Inactive
Add symbolic link at repository root pointing to documentation content in packages/docs/src/content/docs/ to match pattern from infra/ repository. This makes documentation easily discoverable from the repository root while maintaining the monorepo package structure. The symlink uses a relative path for portability across different environments. Also moves existing docs/notes/ directory to packages/docs/src/content/docs/notes/ to preserve working notes within the unified documentation structure. Pattern: docs -> packages/docs/src/content/docs/
Add reusable composite action that queries GitHub Checks API to determine if a job already succeeded for the current commit SHA. Enables per-job intelligent execution decisions based on: - Execution history via GitHub Checks API query - Path-based change detection with configurable regex filters - Force-run override capability This composite action provides the foundation for eliminating centralized workflow coordination jobs and achieving fine-grained per-job and per-matrix-element caching.
Replace centralized skip-check job with per-job intelligent caching using GitHub Checks API. Changes: - Remove skip-check job and fkirc/skip-duplicate-actions dependency - Add force_run workflow input parameter - Update secrets-scan job to use cached-ci-job composite action - Update nix job with path filters for Nix/TypeScript files - Update test job to remove skip-check dependency (matrix caching via API) - Update all job dependencies to eliminate skip-check references - Add fail-fast: false to test job matrix strategy - Add fetch-depth: 0 to jobs using composite action for git diff - Simplify job comments to remove outdated skip-check references Benefits: - 22s faster workflow initialization (eliminate serial helpers) - Per-matrix-element caching granularity - 80% fewer job executions on retry after partial failure - Self-contained job definitions with explicit path filters
cameronraysmith
had a problem deploying
to
preview
GitHub Actions
Failure
Signed-off-by: Cameron Smith <cameron.ray.smith@gmail.com>
Add errorOnLocalLinks: false to starlight-links-validator configuration to permit localhost URLs in development documentation. Root cause: The notes/build/rolldown-workers-incompatibility.md file contains a reference to http://localhost:8787 as part of testing instructions. This is valid for development notes but was being flagged by the link validator. This configuration aligns with typical development documentation patterns where localhost references are acceptable for testing and preview instructions. Fixes link validation error: - Found 1 invalid link in 1 file. - notes/build/rolldown-workers-incompatibility/ - http://localhost:8787 - local link
Upgrade wrangler to match version used in infra repository and eliminate compatibility date warnings. Changes: - Update packages/docs/package.json: wrangler ^4.42.0 → ^4.45.0 - Regenerate worker-configuration.d.ts with improved type definitions - Update bun.lock with wrangler 4.45.3 and dependencies Benefits: - Removes compatibility date warning: "The latest compatibility date supported by the installed Cloudflare Workers Runtime is 2025-10-01, but you've requested 2025-10-06. Falling back to 2025-10-01..." - Improved type generation for Cloudflare Worker environment bindings - Better NodeJS.ProcessEnv type safety for env vars Testing sequence completed successfully: - ✅ just cf-types (wrangler 4.45.3) - ✅ just docs-build - ✅ just docs-test (20 unit + 24 e2e tests passed) - ✅ just docs-linkcheck
cameronraysmith
temporarily deployed
to
preview
GitHub Actions
Inactive
Remove `name: gitleaks` override from secrets-scan job to align with infra repository's naming convention. Rationale: - "secrets-scan" is more generic and describes the job's purpose - "gitleaks" is just the implementation tool (could be swapped) - Consistency between job ID and display name improves clarity - Matches pattern established in infra repository The job ID remains `secrets-scan`, so no changes needed to job dependencies or workflow logic.
…y-critical Expand comment on secrets-scan cache check to explain why caching is both safe and valuable for security-critical jobs. Key insights: 1. Cache improves retry efficiency - don't re-scan on workflow re-run 2. Content-addressed security - commit SHA cryptographically guarantees same content, therefore same security posture 3. Deterministic scanning - gitleaks produces identical results for identical repository content 4. Override available - force_run=true for paranoid manual re-scans The cache WILL skip when: - Workflow retries after partial failure (common scenario) - Manual re-trigger on same commit SHA - Multiple runs on same commit The cache WON'T skip when: - New commits (different SHA = different content) - First run on a commit (no cache entry) - force_run=true override This pattern is established in infra repository and provides significant efficiency gains without compromising security guarantees.
cameronraysmith
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.