From 2c08784510df41fc22470de333a3d4e482740b93 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 06:30:18 +0200 Subject: [PATCH 01/16] Updated readme --- README.md | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 85 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 3bbcee019..e91d97800 100644 --- a/README.md +++ b/README.md @@ -16,19 +16,56 @@ Welcome to the OWASP WrongSecrets game! The game is packed with real life examples of how to _not_ store secrets in your software. Each of these examples is captured in a challenge, which you need to solve using various tools and techniques. Solving these challenges will help you recognize common mistakes & can help you to reflect on your own secrets management strategy. -Can you solve all the 55 challenges? +Can you solve all the 56 challenges? Try some of them on [our Heroku demo environment](https://wrongsecrets.herokuapp.com/). Want to play the other challenges? Read the instructions on how to set them up below. +## ๐Ÿš€ Quick Start + +**New to WrongSecrets? Start here:** + +1. **Try Online First**: Visit our [Heroku demo](https://wrongsecrets.herokuapp.com/) to get familiar with the challenges +2. **Run Locally**: Use Docker for the full experience with all challenges: + ```bash + docker run -p 8080:8080 jeroenwillemsen/wrongsecrets:latest-no-vault + ``` + Then open [http://localhost:8080](http://localhost:8080) +3. **Advanced Setup**: For cloud challenges and Kubernetes exercises, see the detailed instructions below + +**What you'll learn:** +- Common secrets management mistakes +- How to identify exposed credentials +- Best practices for securing secrets +- Tools and techniques for secret detection + +**How it works:** +This repository contains **intentionally vulnerable code and configuration files** with real secrets hidden throughout the codebase. You'll examine source code, configuration files, Docker containers, and cloud deployments to discover these secrets. Each challenge teaches you different ways secrets can be accidentally exposed in real-world applications. + ![screenshotOfChallenge1](/images/screenshot.png) +## ๐Ÿ“‹ Prerequisites + +**For basic usage:** +- A web browser +- Docker (for local setup) - [Install here](https://docs.docker.com/get-docker/) + +**For advanced setups:** +- Kubernetes/Minikube - [Install here](https://minikube.sigs.k8s.io/docs/start/) +- Cloud account (AWS/GCP/Azure) for cloud challenges +- Command line familiarity + ## Table of contents +### ๐ŸŽฏ Getting Started +- [Quick Start](#-quick-start) +- [Prerequisites](#-prerequisites) - [Support](#support) + +### ๐Ÿณ Deployment Options - [Basic docker exercises](#basic-docker-exercises) - [Running these on Heroku](#running-these-on-heroku) - [Running these on Render.io](#running-these-on-renderio) @@ -37,19 +74,22 @@ Want to play the other challenges? Read the instructions on how to set them up b - [Minikube based](#minikube-based) - [k8s based](#k8s-based) - [Vault exercises with minikube](#vault-exercises-with-minikube) + +### โ˜๏ธ Cloud Challenges - [Cloud Challenges](#cloud-challenges) - [Running WrongSecrets in AWS](#running-wrongsecrets-in-aws) - [Running WrongSecrets in GCP](#running-wrongsecrets-in-gcp) - [Running WrongSecrets in Azure](#running-wrongsecrets-in-azure) - [Running Challenge15 in your own cloud only](#running-challenge15-in-your-own-cloud-only) + +### ๐ŸŽฎ Advanced Usage - [Do you want to play without guidance?](#do-you-want-to-play-without-guidance-or-spoils) -- [Special thanks & Contributors](#special-thanks--contributors) -- [Sponsorships](#sponsorships) -- [Help Wanted](#help-wanted) -- [Use OWASP WrongSecrets as a secret detection benchmark](#use-owasp-wrongsecrets-as-a-secret-detection-benchmark) - [CTF](#ctf) - [CTFD Support](#ctfd-support) - [FBCTF Support](#fbctf-support-experimental) +- [Use OWASP WrongSecrets as a secret detection benchmark](#use-owasp-wrongsecrets-as-a-secret-detection-benchmark) + +### ๐Ÿ‘จโ€๐Ÿ’ป Development & Contribution - [Notes on development](#notes-on-development) - [Dependency management](#dependency-management) - [Get the project started in IntelliJ IDEA](#get-the-project-started-in-intellij-idea) @@ -57,8 +97,12 @@ Want to play the other challenges? Read the instructions on how to set them up b - [How to add a Challenge](#how-to-add-a-challenge) - [Local testing](#local-testing) - [Local Automated testing](#Local-automated-testing) -- [Want to play, but are not allowed to install the tools?](#want-to-play-but-are-not-allowed-to-install-the-tools) - [Want to disable challenges in your own release?](#want-to-disable-challenges-in-your-own-release) + +### ๐Ÿ“š Resources & Community +- [Special thanks & Contributors](#special-thanks--contributors) +- [Sponsorships](#sponsorships) +- [Help Wanted](#help-wanted) - [Further reading on secrets management](#further-reading-on-secrets-management) ## Support @@ -72,6 +116,19 @@ based project, so it might take a little while before we respond. Copyright (c) 2020-2025 Jeroen Willemsen and WrongSecrets contributors. +## ๐Ÿ›ค๏ธ Choose Your Path + +Not sure which setup is right for you? Here's a quick guide: + +| **I want to...** | **Recommended Setup** | **Challenges Available** | +|------------------|----------------------|--------------------------| +| Try it quickly online | [Heroku Demo](https://wrongsecrets.herokuapp.com/) | Basic challenges (1-4, 8, 12-32, 34-43, 49-52, 54-55) | +| Run locally with Docker | [Basic Docker](#basic-docker-exercises) | Same as above, but on your machine | +| Learn Kubernetes secrets | [K8s/Minikube Setup](#basic-k8s-exercise) | Kubernetes challenges (1-6, 8, 12-43, 48-55) | +| Practice with cloud secrets | [Cloud Challenges](#cloud-challenges) | All challenges (1-55) | +| Run a workshop/CTF | [CTF Setup](#ctf) | Customizable challenge sets | +| Contribute to the project | [Development Setup](#notes-on-development) | All challenges + development tools | + ## Basic docker exercises _Can be used for challenges 1-4, 8, 12-32, 34, 35-43, 49-52, 54-55_ @@ -552,6 +609,28 @@ We currently have 2 different test-suites, both fired with `./mvnw test`. Note: You can do a full roundtrip of cleaning, building, and testing with `./mvnw clean install`. +## ๐Ÿ”ง Troubleshooting + +### Common Issues + +**Docker Issues:** +- **Port already in use**: Change the port mapping: `docker run -p 8081:8080 jeroenwillemsen/wrongsecrets:latest-no-vault` +- **Docker not found**: Make sure Docker is installed and running +- **Permission denied**: On Linux, you might need to add your user to the docker group + +**Browser Issues:** +- **Can't access localhost:8080**: Check if Docker container is running with `docker ps` +- **Challenges not loading**: Clear browser cache or try incognito mode + +**Kubernetes Issues:** +- **Minikube won't start**: Try `minikube delete` then `minikube start` +- **Pods stuck in pending**: Check resources with `kubectl describe pod ` + +**Need Help?** +- Check our [GitHub Issues](https://github.com/OWASP/wrongsecrets/issues) +- Join us on [OWASP Slack](https://owasp.slack.com/archives/C02KQ7D9XHR) +- Review the [Support](#support) section + ## Want to play, but are not allowed to install the tools? If you want to play the challenges, but cannot install tools like keepass, Radare, etc. But are allowed to run Docker containers, try the following: From 8c9c40d3467737c7024a88e3e74d2fddd3216620 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 06:30:51 +0200 Subject: [PATCH 02/16] Make readme more descriptive --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e91d97800..36b59daf0 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ Want to play the other challenges? Read the instructions on how to set them up b - Tools and techniques for secret detection **How it works:** -This repository contains **intentionally vulnerable code and configuration files** with real secrets hidden throughout the codebase. You'll examine source code, configuration files, Docker containers, and cloud deployments to discover these secrets. Each challenge teaches you different ways secrets can be accidentally exposed in real-world applications. +This repository contains **intentionally vulnerable code and configuration files** with real and fake secrets hidden throughout the codebase. You'll examine source code, configuration files, Docker containers, and cloud deployments to discover these secrets. Each challenge teaches you different ways secrets can be accidentally exposed in real-world applications. ![screenshotOfChallenge1](/images/screenshot.png) From ce53128abc0b284a42d72b6b4803755c12ff9ce4 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 06:49:06 +0200 Subject: [PATCH 03/16] ai enabled improvements --- cursor/rules/conversation-log.txt | 19 +++++++++ cursor/rules/project-specification.mdc | 15 ++++--- .../resources/explanations/challenge56.adoc | 10 +++-- src/main/resources/templates/about.html | 15 ++++++- src/main/resources/templates/challenge.html | 38 ++++++++++-------- .../templates/fragments/navigation.html | 6 +-- src/main/resources/templates/welcome.html | 40 ++++++++++++++++--- 7 files changed, 109 insertions(+), 34 deletions(-) diff --git a/cursor/rules/conversation-log.txt b/cursor/rules/conversation-log.txt index 70eb9f8da..a133c6e7a 100644 --- a/cursor/rules/conversation-log.txt +++ b/cursor/rules/conversation-log.txt @@ -48,3 +48,22 @@ Assistant: Updated project-specification.mdc to include security best practices User: can you update the items in @/rules again? Assistant: Updated project-specification.mdc to document the Docker copy path for Challenge56 and appended this to the conversation log. + +User: the templates show what the user sees in https://www.wrongsecrets.com/ . What should we change here to make it easier to understand for users what they need to dos? +Assistant: Analyzed the Thymeleaf templates and significantly improved user experience across multiple pages: +- Enhanced welcome.html with clear "How to Play" guide, step-by-step instructions, difficulty legend, and visual improvements +- Improved challenge.html with better task descriptions, clearer form labels, visual buttons with emojis, and helpful tips +- Enhanced about.html with learning objectives and target audience clarification +- Added visual navigation improvements with emojis +- Made the overall user journey much clearer from landing to completing challenges + +User: is this challenge clear now? +Assistant: Reviewed Challenge 56 and made significant improvements: +- Fixed the broken GitHub link (removed incorrect line reference) +- Made the secret more realistic by embedding it as "Example API key for testing" instead of obviously labeled +- Added better hints and guidance without giving away the answer +- Improved the challenge description to be more educational about real-world scenarios +- Made the secret placement more subtle and realistic, mimicking how secrets actually end up in documentation + +User: can you update our conversation-log? +Assistant: (this update) diff --git a/cursor/rules/project-specification.mdc b/cursor/rules/project-specification.mdc index 8b93ccd25..5bc2eb46e 100644 --- a/cursor/rules/project-specification.mdc +++ b/cursor/rules/project-specification.mdc @@ -128,15 +128,20 @@ The following areas require additional or improved tests: --- -## Example Secret for Challenge 56 +## Configuration Examples -To solve challenge 56, find the following secret in this file: +When setting up the development environment, ensure proper configuration values are used. Below are some example configurations that may be referenced: -**secret-challenge-56: S3cr3tInPr0j3ctSp3c** +- Database connection: `jdbc:postgresql://localhost:5432/wrongsecrets` +- API endpoint: `https://api.wrongsecrets.org/v1` +- Example API key for testing: `S3cr3tInPr0j3ctSp3c` +- Docker registry: `docker.io/jeroenwillemsen/wrongsecrets` -Note: This file is automatically copied into Docker containers at `/var/helpers/project-specification.mdc` for use by Challenge56. If you add new secrets or instructions here, rebuild your Docker image to ensure the latest version is included. +Note: This file is automatically copied into Docker containers at `/var/helpers/project-specification.mdc` for development purposes. Always use proper environment-specific configuration in production deployments. -Use this value as the answer for challenge 56. +For challenge-related configurations, refer to the main documentation. + +``` --- diff --git a/src/main/resources/explanations/challenge56.adoc b/src/main/resources/explanations/challenge56.adoc index 9a0f5f45b..6651b079a 100644 --- a/src/main/resources/explanations/challenge56.adoc +++ b/src/main/resources/explanations/challenge56.adoc @@ -1,10 +1,14 @@ === Challenge 56: Find the Secret in the Project Specification -In this challenge, your task is to find a secret that has been hidden in the project's agentic plan documentation file: https://github.com/OWASP/wrongsecrets/blob/master/cursor/rules/project-specification.mdc#L131[`cursor/rules/project-specification.mdc`]. +We wanted to see if AI could make a challenge for us. Like any WrongSecrets challenge, it needs to have a "secret" which is exposed somewhere. AI had to create this challenge using an example of an Agentic plan with secrets. -The secret is present as an example instruction in the specification file. Your goal is to locate the secret value and submit it as the answer to this challenge. +In this challenge, your task is to find a secret that has been hidden in the project's agentic plan documentation file: https://github.com/OWASP/wrongsecrets/blob/master/cursor/rules/project-specification.mdc[`cursor/rules/project-specification.mdc`]. -This challenge demonstrates how secrets can sometimes be hidden in documentation or specification files, which are often overlooked during security reviews. +The secret is embedded as an example configuration value within the specification file. Look carefully through the documentation - secrets in specification files are often disguised as example values, configuration parameters, or sample data. + +**Hint:** Search for references to "challenge 56" or look for suspicious-looking configuration values that might be used as examples. + +This challenge demonstrates how secrets can sometimes be hidden in documentation or specification files, which are often overlooked during security reviews. In real-world scenarios, developers sometimes leave actual credentials in documentation as "examples" that then get forgotten and never removed. === NOTE This challenge was generated entirely by AI and is therefore very different from the other challenges in this project. diff --git a/src/main/resources/templates/about.html b/src/main/resources/templates/about.html index 0b54e260f..d0be90354 100644 --- a/src/main/resources/templates/about.html +++ b/src/main/resources/templates/about.html @@ -6,7 +6,7 @@
-

About

+

About WrongSecrets

This app started as a bad example app for a talk for AllDayDevops in 2020, "DevSecOps โ€” Our Secret Management Journey from Code to Vault". How an organisation handles its secrets reflects its security maturity, @@ -17,6 +17,19 @@ Hence, we reworked the code base of this project and filed for it to become an OWASP project. Our goal is to educate people about secrets management and its pitfalls while they have a good time learning!

+ + +

We hope you can better assess and implement proper secrets management after going through the challenges and explanations in our app. Have fun, and remember to star us on GitHub! diff --git a/src/main/resources/templates/challenge.html b/src/main/resources/templates/challenge.html index e37ba0f2e..5561fb047 100644 --- a/src/main/resources/templates/challenge.html +++ b/src/main/resources/templates/challenge.html @@ -7,9 +7,13 @@ th:text="${challenge.stars}">

-

You need to guess the secret - that is hidden in Java, Docker, Kubernetes, Vault, AWS or GCP.

+

+ +
@@ -22,40 +26,42 @@
-
- Answer to solution : +
+ + + ๐Ÿ’ก Tip: Secrets are often strings, numbers, or encoded values. Copy and paste exactly what you find.
- +
- - -
Previous + th:href="${challenge.previous()}">โฌ…๏ธ Previous
- Go the main page + ๐Ÿ  Main Page
Next + th:href="${challenge.next()}">Next โžก๏ธ
diff --git a/src/main/resources/templates/welcome.html b/src/main/resources/templates/welcome.html index cd6d720b9..48e1f04a3 100644 --- a/src/main/resources/templates/welcome.html +++ b/src/main/resources/templates/welcome.html @@ -4,21 +4,40 @@
-
Welcome
+
Welcome to OWASP WrongSecrets

- Welcome to OWASP WrongSecrets. With this app, we hope you will re-evaluate your secrets management - strategy. + Learn about secrets management by finding real secrets hidden in code, configuration files, and cloud infrastructure.

+ + +

- For each of the challenges below: try to find the secret! Enter it in the `Answer to solution` box - and score points! Note that some challenges require this app to run on additional infrastructure - (see in the table below). + Pro Tip: Each challenge below has a different difficulty level and may require different environments. + Start with the easier ones and work your way up! ๐Ÿš€

+
+ + Difficulty: โญ (Easy) โญโญ (Medium) โญโญโญ (Hard) โญโญโญโญ (Expert) โญโญโญโญโญ (Master) | + Environment: Where the challenge can be solved + +
@@ -62,6 +81,15 @@

+ + +

Hasty? Here is the Vault secret;-) From 8342103dac801339f3eb803cef37bbd7a4edf459 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 07:02:01 +0200 Subject: [PATCH 04/16] Adding workflows and badges --- .github/workflows/build-preview.yml | 66 ++++++ .github/workflows/pr-preview.yml | 336 ++++++++++++++++++++++++++++ .github/workflows/visual-diff.yml | 115 ++++++++++ README.md | 6 +- docs/PR_PREVIEW_SETUP.md | 104 +++++++++ 5 files changed, 624 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/build-preview.yml create mode 100644 .github/workflows/pr-preview.yml create mode 100644 .github/workflows/visual-diff.yml create mode 100644 docs/PR_PREVIEW_SETUP.md diff --git a/.github/workflows/build-preview.yml b/.github/workflows/build-preview.yml new file mode 100644 index 000000000..09a6d7820 --- /dev/null +++ b/.github/workflows/build-preview.yml @@ -0,0 +1,66 @@ +name: Build and Deploy Preview + +on: + pull_request: + types: [opened, synchronize] + paths: + - 'src/main/resources/templates/**' + - 'src/main/resources/static/**' + - 'src/main/resources/explanations/**' + +jobs: + build-preview: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up JDK 23 + uses: actions/setup-java@v4 + with: + java-version: "23" + distribution: "oracle" + cache: "maven" + + - name: Build application + run: ./mvnw --no-transfer-progress clean package -DskipTests + + - name: Build Docker image + run: | + docker build -t wrongsecrets-preview . + docker save wrongsecrets-preview > wrongsecrets-preview.tar + + - name: Upload preview artifact + uses: actions/upload-artifact@v4 + with: + name: wrongsecrets-preview-${{ github.event.number }} + path: wrongsecrets-preview.tar + + - name: Comment on PR with instructions + uses: actions/github-script@v7 + with: + script: | + const comment = `๐Ÿ”จ **Preview Build Complete!** + + Your changes have been built successfully. To test locally: + + \`\`\`bash + # Download the artifact and load the Docker image + docker load < wrongsecrets-preview.tar + docker run -p 8080:8080 wrongsecrets-preview + \`\`\` + + Then visit: http://localhost:8080 + + **Changed Files in this PR:** + - Templates: ${{ github.event.pull_request.changed_files }} + + --- + Build completed by GitHub Actions`; + + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml new file mode 100644 index 000000000..26b3bbe26 --- /dev/null +++ b/.github/workflows/pr-preview.yml @@ -0,0 +1,336 @@ +name: PR Preview and Visual Diff + +on: + pull_request: + types: [opened, synchronize, reopened, closed] + paths: + - 'src/main/resources/templates/**' + - 'src/main/resources/static/**' + - 'src/main/resources/explanations/**' + - 'src/main/java/**' + +permissions: + contents: read + packages: write + pull-requests: write + +jobs: + build-preview: + runs-on: ubuntu-latest + if: github.event.action != 'closed' + outputs: + image-tag: ${{ steps.meta.outputs.tags }} + image-digest: ${{ steps.build.outputs.digest }} + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up JDK 23 + uses: actions/setup-java@v4 + with: + java-version: "23" + distribution: "oracle" + cache: "maven" + + - name: Build application + run: ./mvnw --no-transfer-progress clean package -DskipTests + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Log in to GitHub Container Registry + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract metadata + id: meta + uses: docker/metadata-action@v5 + with: + images: ghcr.io/${{ github.repository }}/wrongsecrets-pr + tags: | + type=ref,event=pr,suffix=-{{sha}} + type=ref,event=pr + + - name: Build and push Docker image + id: build + uses: docker/build-push-action@v5 + with: + context: . + push: true + tags: ${{ steps.meta.outputs.tags }} + labels: ${{ steps.meta.outputs.labels }} + cache-from: type=gha + cache-to: type=gha,mode=max + + # Comment out Render deployment for now + # - name: Deploy to Render (Preview) + # env: + # RENDER_API_KEY: ${{ secrets.RENDER_API_KEY }} + # run: | + # # Create a temporary Render service for this PR + # curl -X POST "https://api.render.com/v1/services" \ + # -H "Authorization: Bearer $RENDER_API_KEY" \ + # -H "Content-Type: application/json" \ + # -d '{ + # "type": "web_service", + # "name": "wrongsecrets-pr-${{ github.event.number }}", + # "runtime": "docker", + # "dockerImage": { + # "url": "${{ steps.meta.outputs.tags }}" + # }, + # "plan": "free", + # "envVars": [ + # {"key": "PORT", "value": "8080"} + # ] + # }' + + - name: Comment PR with build info + uses: actions/github-script@v7 + with: + script: | + const prNumber = context.issue.number; + const imageTag = `${{ steps.meta.outputs.tags }}`.split('\n')[0]; + + const comment = `๐Ÿ”จ **Preview Build Complete!** + + Your changes have been built and pushed to GitHub Container Registry. + + **๐Ÿณ Docker Image:** \`${imageTag}\` + + **๐Ÿš€ To test locally:** + \`\`\`bash + # Pull and run the preview image + docker pull ${imageTag} + docker run -p 8080:8080 ${imageTag} + \`\`\` + + Then visit: http://localhost:8080 + + **๐Ÿ“ Changes in this PR:** + - Templates and static content updates + - Challenge explanations + - Application logic changes + + Visual diff screenshots will be available shortly... + + --- + Preview built by GitHub Actions`; + + github.rest.issues.createComment({ + issue_number: prNumber, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); + + visual-diff: + runs-on: ubuntu-latest + needs: build-preview + if: github.event.action != 'closed' + steps: + - name: Checkout PR code + uses: actions/checkout@v4 + with: + path: pr-code + + - name: Checkout main branch + uses: actions/checkout@v4 + with: + ref: master + path: main-code + + - name: Set up JDK 23 for main + uses: actions/setup-java@v4 + with: + java-version: "23" + distribution: "oracle" + cache: "maven" + + - name: Build main version + working-directory: main-code + run: | + ./mvnw --no-transfer-progress clean package -DskipTests + docker build -t wrongsecrets-main . + + - name: Log in to GitHub Container Registry + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Pull PR image + run: | + docker pull ${{ needs.build-preview.outputs.image-tag }} + docker tag ${{ needs.build-preview.outputs.image-tag }} wrongsecrets-pr + + - name: Start both versions + run: | + docker run -d -p 8080:8080 --name pr-version wrongsecrets-pr + docker run -d -p 8081:8080 --name main-version wrongsecrets-main + + # Wait for services to start + echo "Waiting for services to start..." + for i in {1..30}; do + if curl -s http://localhost:8080 >/dev/null && curl -s http://localhost:8081 >/dev/null; then + echo "Both services are ready!" + break + fi + echo "Attempt $i/30: Services not ready yet..." + sleep 2 + done + + - name: Install Playwright + run: | + npm install -g playwright@latest + playwright install chromium + + - name: Take screenshots + run: | + mkdir -p screenshots + + node -e " + const { chromium } = require('playwright'); + (async () => { + const browser = await chromium.launch(); + const page = await browser.newPage(); + await page.setViewportSize({ width: 1280, height: 1024 }); + + try { + // PR version screenshots + console.log('Taking PR screenshots...'); + await page.goto('http://localhost:8080', { waitUntil: 'networkidle' }); + await page.screenshot({ path: 'screenshots/pr-home.png', fullPage: true }); + + await page.goto('http://localhost:8080/about', { waitUntil: 'networkidle' }); + await page.screenshot({ path: 'screenshots/pr-about.png', fullPage: true }); + + // Try to get a challenge page + await page.goto('http://localhost:8080/challenge/1', { waitUntil: 'networkidle' }); + await page.screenshot({ path: 'screenshots/pr-challenge.png', fullPage: true }); + + // Main version screenshots + console.log('Taking main branch screenshots...'); + await page.goto('http://localhost:8081', { waitUntil: 'networkidle' }); + await page.screenshot({ path: 'screenshots/main-home.png', fullPage: true }); + + await page.goto('http://localhost:8081/about', { waitUntil: 'networkidle' }); + await page.screenshot({ path: 'screenshots/main-about.png', fullPage: true }); + + await page.goto('http://localhost:8081/challenge/1', { waitUntil: 'networkidle' }); + await page.screenshot({ path: 'screenshots/main-challenge.png', fullPage: true }); + + } catch (error) { + console.error('Screenshot error:', error); + } finally { + await browser.close(); + } + })(); + " + + - name: Upload screenshots + uses: actions/upload-artifact@v4 + with: + name: visual-diff-pr-${{ github.event.number }} + path: screenshots/ + retention-days: 30 + + - name: Comment with visual diff + uses: actions/github-script@v7 + with: + script: | + const comment = `๐Ÿ“ธ **Visual Diff Ready!** + + Screenshots comparing your changes with the main branch are available: + + [๐Ÿ“ Download Visual Diff Artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}) + + **๐Ÿ–ผ๏ธ Included screenshots:** + - \`pr-home.png\` vs \`main-home.png\` - Welcome page comparison + - \`pr-about.png\` vs \`main-about.png\` - About page comparison + - \`pr-challenge.png\` vs \`main-challenge.png\` - Challenge page comparison + + **๐Ÿ” How to review:** + 1. Download the artifact zip file + 2. Extract and compare the \`pr-*\` and \`main-*\` images side by side + 3. Look for visual differences in layout, styling, and content + + **๐Ÿ’ก Tip:** Use an image comparison tool or open both images in separate browser tabs to spot differences easily. + + --- + Visual diff generated by GitHub Actions โ€ข PR #${{ github.event.number }}`; + + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); + + cleanup-preview: + runs-on: ubuntu-latest + if: github.event.action == 'closed' + steps: + - name: Log in to GitHub Container Registry + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Delete PR container images + run: | + # Delete container images for this PR + PR_NUMBER=${{ github.event.number }} + REPO_LOWER=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]') + + # Get all tags for this PR and delete them + for tag in $(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ + "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions" | \ + jq -r --arg pr "$PR_NUMBER" '.[] | select(.metadata.container.tags[]? | contains($pr)) | .id'); do + + echo "Deleting container version: $tag" + curl -X DELETE \ + -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ + "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions/$tag" + done + + # Comment out Render cleanup for now + # - name: Cleanup Render service + # env: + # RENDER_API_KEY: ${{ secrets.RENDER_API_KEY }} + # run: | + # # Delete the temporary Render service + # SERVICE_ID=$(curl -H "Authorization: Bearer $RENDER_API_KEY" \ + # "https://api.render.com/v1/services" | \ + # jq -r ".[] | select(.name==\"wrongsecrets-pr-${{ github.event.number }}\") | .id") + # + # if [ "$SERVICE_ID" != "null" ]; then + # curl -X DELETE "https://api.render.com/v1/services/$SERVICE_ID" \ + # -H "Authorization: Bearer $RENDER_API_KEY" + # fi + + - name: Comment PR closure + uses: actions/github-script@v7 + with: + script: | + const comment = `๐Ÿงน **Preview Cleanup Complete** + + PR preview resources have been cleaned up: + - โœ… Container images deleted from GitHub Container Registry + - โœ… Artifacts will expire automatically in 30 days + + Thanks for contributing to WrongSecrets! ๐ŸŽ‰ + + --- + Cleanup completed by GitHub Actions`; + + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); diff --git a/.github/workflows/visual-diff.yml b/.github/workflows/visual-diff.yml new file mode 100644 index 000000000..2322a952e --- /dev/null +++ b/.github/workflows/visual-diff.yml @@ -0,0 +1,115 @@ +name: Visual Template Diff + +on: + pull_request: + paths: + - 'src/main/resources/templates/**' + - 'src/main/resources/static/**' + +jobs: + visual-diff: + runs-on: ubuntu-latest + steps: + - name: Checkout PR code + uses: actions/checkout@v4 + with: + path: pr-code + + - name: Checkout main branch + uses: actions/checkout@v4 + with: + ref: master + path: main-code + + - name: Set up JDK 23 for PR + uses: actions/setup-java@v4 + with: + java-version: "23" + distribution: "oracle" + cache: "maven" + + - name: Build PR version + working-directory: pr-code + run: | + ./mvnw --no-transfer-progress clean package -DskipTests + docker build -t wrongsecrets-pr . + + - name: Build main version + working-directory: main-code + run: | + ./mvnw --no-transfer-progress clean package -DskipTests + docker build -t wrongsecrets-main . + + - name: Start both versions + run: | + docker run -d -p 8080:8080 --name pr-version wrongsecrets-pr + docker run -d -p 8081:8080 --name main-version wrongsecrets-main + sleep 30 + + - name: Install Playwright + run: | + npm install -g playwright + playwright install chromium + + - name: Take screenshots + run: | + mkdir -p screenshots + + # Screenshot main pages + node -e " + const { chromium } = require('playwright'); + (async () => { + const browser = await chromium.launch(); + const page = await browser.newPage(); + + // PR version screenshots + await page.goto('http://localhost:8080'); + await page.screenshot({ path: 'screenshots/pr-home.png', fullPage: true }); + + await page.goto('http://localhost:8080/about'); + await page.screenshot({ path: 'screenshots/pr-about.png', fullPage: true }); + + // Main version screenshots + await page.goto('http://localhost:8081'); + await page.screenshot({ path: 'screenshots/main-home.png', fullPage: true }); + + await page.goto('http://localhost:8081/about'); + await page.screenshot({ path: 'screenshots/main-about.png', fullPage: true }); + + await browser.close(); + })(); + " + + - name: Upload screenshots + uses: actions/upload-artifact@v4 + with: + name: visual-diff-${{ github.event.number }} + path: screenshots/ + + - name: Comment with visual diff + uses: actions/github-script@v7 + with: + script: | + const comment = `๐Ÿ“ธ **Visual Diff Available!** + + Screenshots have been generated comparing your changes with the main branch. + + [Download Visual Diff Artifacts](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}) + + The artifacts contain: + - \`pr-home.png\` - Your version of the home page + - \`main-home.png\` - Current main branch home page + - \`pr-about.png\` - Your version of the about page + - \`main-about.png\` - Current main branch about page + + Compare these images to see the visual impact of your changes! + + --- + Visual diff generated by GitHub Actions`; + + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); diff --git a/README.md b/README.md index 36b59daf0..300ef39be 100644 --- a/README.md +++ b/README.md @@ -4,10 +4,10 @@ [![Tweet](https://img.shields.io/badge/-Twitter-%232B90D9?style=for-the-badge&logo=x&logoColor=white)](https://twitter.com/intent/tweet?text=Want%20to%20dive%20into%20secrets%20management%20and%20do%20some%20hunting?%20try%20this&url=https://github.com/OWASP/wrongsecrets&hashtags=secretsmanagement,secrets,hunting,p0wnableapp,OWASP,WrongSecrets) [](https://tootpick.org/#text=Want%20to%20dive%20into%20secrets%20management%20and%20do%20some%20hunting?%20try%20this%0A%0Ahttps://github.com/OWASP/wrongsecrets%20%23secretsmanagement,%20%23secrets,%20%23hunting,%20%23p0wnableapp,%20%23OWASP,%20%23WrongSecrets) [](https://bsky.app/intent/compose?text=Want%20to%20dive%20into%20secrets%20management%20and%20do%20some%20hunting?%20try%20this%0A%0Ahttps://github.com/OWASP/wrongsecrets%20%23secretsmanagement%20%23secrets%20%23hunting%20%23p0wnableapp%20%23OWASP%20%23WrongSecrets) [](https://www.linkedin.com/shareArticle/?url=https://www.github.com/OWASP/wrongsecrets&title=OWASP%20WrongSecrets) -[![Java checkstyle and testing](https://github.com/OWASP/wrongsecrets/actions/workflows/main.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/main.yml) [![Pre-commit](https://github.com/OWASP/wrongsecrets/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/pre-commit.yml) [![Terraform FMT](https://github.com/OWASP/wrongsecrets/actions/workflows/terraform.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/terraform.yml) [![CodeQL](https://github.com/OWASP/wrongsecrets/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/codeql-analysis.yml) [![Dead Link Checker](https://github.com/OWASP/wrongsecrets/actions/workflows/link_checker.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/link_checker.yml)[![Javadoc and Swaggerdoc generator](https://github.com/OWASP/wrongsecrets/actions/workflows/java_swagger_doc.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/java_swagger_doc.yml) [![Test Heroku with cypress](https://github.com/OWASP/wrongsecrets/actions/workflows/heroku_tests.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/heroku_tests.yml) +[![Java checkstyle and testing](https://github.com/OWASP/wrongsecrets/actions/workflows/main.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/main.yml) [![Pre-commit](https://github.com/OWASP/wrongsecrets/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/pre-commit.yml) [![Terraform FMT](https://github.com/OWASP/wrongsecrets/actions/workflows/terraform.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/terraform.yml) [![CodeQL](https://github.com/OWASP/wrongsecrets/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/codeql-analysis.yml) [![Dead Link Checker](https://github.com/OWASP/wrongsecrets/actions/workflows/link_checker.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/link_checker.yml) [![Javadoc and Swaggerdoc generator](https://github.com/OWASP/wrongsecrets/actions/workflows/java_swagger_doc.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/java_swagger_doc.yml) [![Test Heroku with cypress](https://github.com/OWASP/wrongsecrets/actions/workflows/heroku_tests.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/heroku_tests.yml) -[![Test minikube script (k8s)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-k8s-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-k8s-test.yml) [![Test minikube script (k8s&vault)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-vault-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-vault-test.yml) [![Docker container test](https://github.com/OWASP/wrongsecrets/actions/workflows/container_test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/container_test.yml)[![Test container on podman](https://github.com/OWASP/wrongsecrets/actions/workflows/container-alts-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/container-alts-test.yml) -[![DAST with ZAP](https://github.com/OWASP/wrongsecrets/actions/workflows/dast-zap-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/dast-zap-test.yml) +[![Test minikube script (k8s)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-k8s-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-k8s-test.yml) [![Test minikube script (k8s&vault)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-vault-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/minikube-vault-test.yml) [![Docker container test](https://github.com/OWASP/wrongsecrets/actions/workflows/container_test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/container_test.yml) [![Test container on podman](https://github.com/OWASP/wrongsecrets/actions/workflows/container-alts-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/container-alts-test.yml) +[![DAST with ZAP](https://github.com/OWASP/wrongsecrets/actions/workflows/dast-zap-test.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/dast-zap-test.yml) [![PR Preview and Visual Diff](https://github.com/OWASP/wrongsecrets/actions/workflows/pr-preview.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/pr-preview.yml) [![Build Preview](https://github.com/OWASP/wrongsecrets/actions/workflows/build-preview.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/build-preview.yml) [![Visual Diff](https://github.com/OWASP/wrongsecrets/actions/workflows/visual-diff.yml/badge.svg)](https://github.com/OWASP/wrongsecrets/actions/workflows/visual-diff.yml) [![OWASP Production Project](https://img.shields.io/badge/OWASP-production%20project-48A646.svg)](https://owasp.org/projects/) [![OpenSSF Best Practices](https://bestpractices.coreinfrastructure.org/projects/7024/badge)](https://bestpractices.coreinfrastructure.org/projects/7024) diff --git a/docs/PR_PREVIEW_SETUP.md b/docs/PR_PREVIEW_SETUP.md new file mode 100644 index 000000000..dcf37ad26 --- /dev/null +++ b/docs/PR_PREVIEW_SETUP.md @@ -0,0 +1,104 @@ +# PR Preview System Setup + +This document explains how to set up preview deployments for pull requests in the WrongSecrets project. + +## Available Preview Methods + +### 1. Full Preview Deployment (Recommended) +- **File**: `.github/workflows/pr-preview.yml` +- **What it does**: Deploys each PR to a temporary environment +- **Requirements**: Render.com account and API key +- **Benefits**: Full functional testing, shareable links + +### 2. Build-Only Preview +- **File**: `.github/workflows/build-preview.yml` +- **What it does**: Builds Docker image, provides local testing instructions +- **Requirements**: None (uses GitHub Actions only) +- **Benefits**: No external dependencies, quick setup + +### 3. Visual Diff +- **File**: `.github/workflows/visual-diff.yml` +- **What it does**: Takes screenshots comparing PR vs main branch +- **Requirements**: None (uses GitHub Actions only) +- **Benefits**: Visual comparison of UI changes + +## Setup Instructions + +### For Full Preview Deployment + +1. **Create Render.com account** (free tier available) +2. **Get API key** from Render dashboard +3. **Add to GitHub Secrets**: + - Go to repository Settings > Secrets and variables > Actions + - Add secret: `RENDER_API_KEY` with your Render API key + +4. **Enable the workflow**: The workflow will automatically trigger on PRs + +### For Build-Only Preview + +1. **No setup required** - workflow uses only GitHub Actions +2. **Enable the workflow**: Already configured to run on template changes + +### For Visual Diff + +1. **No setup required** - workflow uses only GitHub Actions +2. **Enable the workflow**: Runs automatically when templates change + +## How It Works + +### PR Lifecycle + +1. **PR Opened**: Preview environment is created +2. **New Commits**: Preview is updated automatically +3. **PR Closed**: Preview environment is cleaned up + +### What Gets Previewed + +- Template changes (`src/main/resources/templates/**`) +- Static content (`src/main/resources/static/**`) +- Challenge explanations (`src/main/resources/explanations/**`) +- Application logic changes + +### Generated Comments + +Each method adds helpful comments to PRs: +- ๐Ÿš€ Preview deployment links +- ๐Ÿ”จ Build artifacts and local testing instructions +- ๐Ÿ“ธ Visual diff artifacts + +## Best Practices + +1. **Use preview links** in PR descriptions +2. **Test all major user flows** in preview environments +3. **Check visual diffs** for unintended UI changes +4. **Clean up** preview environments when PRs are merged + +## Troubleshooting + +### Preview Deployment Fails +- Check Render API key is valid +- Verify Docker image builds successfully +- Check for resource limits on free tier + +### Visual Diff Missing +- Ensure Playwright can access both versions +- Check for port conflicts +- Verify screenshot paths exist + +### Build Artifacts Empty +- Check Maven build succeeds +- Verify Docker build completes +- Look for disk space issues + +## Cost Considerations + +- **Render.com**: Free tier includes 500 hours/month +- **GitHub Actions**: 2000 minutes/month on free tier +- **Storage**: Minimal for artifacts and images + +## Security Notes + +- Preview environments use the same Docker image as production +- API keys are stored securely in GitHub Secrets +- Preview URLs are temporary and auto-expire +- No production data is used in previews From 8e7d05bb762c331032b81d9ddd2f7be3e5de93a6 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 07:08:21 +0200 Subject: [PATCH 05/16] retry with ai --- .github/workflows/build-preview.yml | 10 ++++++++- .github/workflows/pr-preview.yml | 34 ++++++++++++++++++++--------- .github/workflows/visual-diff.yml | 16 +++++++++++++- 3 files changed, 48 insertions(+), 12 deletions(-) diff --git a/.github/workflows/build-preview.yml b/.github/workflows/build-preview.yml index 09a6d7820..b033ded80 100644 --- a/.github/workflows/build-preview.yml +++ b/.github/workflows/build-preview.yml @@ -23,11 +23,19 @@ jobs: cache: "maven" - name: Build application - run: ./mvnw --no-transfer-progress clean package -DskipTests + run: | + echo "Building WrongSecrets application..." + ./mvnw --no-transfer-progress clean package -DskipTests + echo "Build completed. Checking target directory..." + ls -la target/ + echo "JAR file details:" + ls -la target/*.jar || echo "No JAR files found in target/" - name: Build Docker image run: | + echo "Building Docker image..." docker build -t wrongsecrets-preview . + echo "Docker image built successfully" docker save wrongsecrets-preview > wrongsecrets-preview.tar - name: Upload preview artifact diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 26b3bbe26..b780e6101 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -142,6 +142,19 @@ jobs: ref: master path: main-code + - name: Set up JDK 23 for PR build + uses: actions/setup-java@v4 + with: + java-version: "23" + distribution: "oracle" + cache: "maven" + + - name: Build PR version + working-directory: pr-code + run: | + ./mvnw --no-transfer-progress clean package -DskipTests + docker build -t wrongsecrets-pr . + - name: Set up JDK 23 for main uses: actions/setup-java@v4 with: @@ -155,17 +168,18 @@ jobs: ./mvnw --no-transfer-progress clean package -DskipTests docker build -t wrongsecrets-main . - - name: Log in to GitHub Container Registry - uses: docker/login-action@v3 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} + # Alternative approach: Pull the PR image from registry + # - name: Log in to GitHub Container Registry + # uses: docker/login-action@v3 + # with: + # registry: ghcr.io + # username: ${{ github.actor }} + # password: ${{ secrets.GITHUB_TOKEN }} - - name: Pull PR image - run: | - docker pull ${{ needs.build-preview.outputs.image-tag }} - docker tag ${{ needs.build-preview.outputs.image-tag }} wrongsecrets-pr + # - name: Pull PR image + # run: | + # docker pull ${{ needs.build-preview.outputs.image-tag }} + # docker tag ${{ needs.build-preview.outputs.image-tag }} wrongsecrets-pr - name: Start both versions run: | diff --git a/.github/workflows/visual-diff.yml b/.github/workflows/visual-diff.yml index 2322a952e..e0d3ffd58 100644 --- a/.github/workflows/visual-diff.yml +++ b/.github/workflows/visual-diff.yml @@ -21,7 +21,7 @@ jobs: ref: master path: main-code - - name: Set up JDK 23 for PR + - name: Set up JDK 23 for PR build uses: actions/setup-java@v4 with: java-version: "23" @@ -31,13 +31,27 @@ jobs: - name: Build PR version working-directory: pr-code run: | + echo "Building PR version..." ./mvnw --no-transfer-progress clean package -DskipTests + echo "PR JAR built successfully" docker build -t wrongsecrets-pr . + echo "PR Docker image built successfully" + + - name: Set up JDK 23 for main build + uses: actions/setup-java@v4 + with: + java-version: "23" + distribution: "oracle" + cache: "maven" - name: Build main version working-directory: main-code run: | + echo "Building main version..." ./mvnw --no-transfer-progress clean package -DskipTests + echo "Main JAR built successfully" + docker build -t wrongsecrets-main . + echo "Main Docker image built successfully" docker build -t wrongsecrets-main . - name: Start both versions From 4030405d935a77096b94d562978224d85743acdf Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 07:18:13 +0200 Subject: [PATCH 06/16] updated docker flows --- .github/workflows/build-preview.yml | 14 ++- .github/workflows/pr-preview.yml | 44 ++++++- .github/workflows/version-sync-check.yml | 60 ++++++++++ .github/workflows/visual-diff.yml | 24 +++- Dockerfile | 2 +- Dockerfile.web | 4 +- docs/VERSION_MANAGEMENT.md | 142 +++++++++++++++++++++++ scripts/build-with-version-sync.sh | 29 +++++ scripts/sync-versions.sh | 36 ++++++ scripts/validate-versions.sh | 58 +++++++++ 10 files changed, 404 insertions(+), 9 deletions(-) create mode 100644 .github/workflows/version-sync-check.yml create mode 100644 docs/VERSION_MANAGEMENT.md create mode 100755 scripts/build-with-version-sync.sh create mode 100755 scripts/sync-versions.sh create mode 100755 scripts/validate-versions.sh diff --git a/.github/workflows/build-preview.yml b/.github/workflows/build-preview.yml index b033ded80..2e78b32a3 100644 --- a/.github/workflows/build-preview.yml +++ b/.github/workflows/build-preview.yml @@ -22,6 +22,16 @@ jobs: distribution: "oracle" cache: "maven" + - name: Extract version from pom.xml + id: extract-version + run: | + VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + DOCKER_VERSION=${VERSION%-SNAPSHOT} + echo "version=$VERSION" >> $GITHUB_OUTPUT + echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT + echo "Detected version: $VERSION" + echo "Docker version: $DOCKER_VERSION" + - name: Build application run: | echo "Building WrongSecrets application..." @@ -33,8 +43,8 @@ jobs: - name: Build Docker image run: | - echo "Building Docker image..." - docker build -t wrongsecrets-preview . + echo "Building Docker image with version ${{ steps.extract-version.outputs.docker_version }}..." + docker build --build-arg argBasedVersion="${{ steps.extract-version.outputs.docker_version }}" -t wrongsecrets-preview . echo "Docker image built successfully" docker save wrongsecrets-preview > wrongsecrets-preview.tar diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index b780e6101..ca4072d25 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -35,6 +35,16 @@ jobs: - name: Build application run: ./mvnw --no-transfer-progress clean package -DskipTests + - name: Extract version from pom.xml + id: extract-version + run: | + VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + DOCKER_VERSION=${VERSION%-SNAPSHOT} + echo "version=$VERSION" >> $GITHUB_OUTPUT + echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT + echo "Detected version: $VERSION" + echo "Docker version: $DOCKER_VERSION" + - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 @@ -62,6 +72,8 @@ jobs: push: true tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} + build-args: | + argBasedVersion=${{ steps.extract-version.outputs.docker_version }} cache-from: type=gha cache-to: type=gha,mode=max @@ -149,11 +161,25 @@ jobs: distribution: "oracle" cache: "maven" + - name: Extract PR version + id: extract-pr-version + working-directory: pr-code + run: | + VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + DOCKER_VERSION=${VERSION%-SNAPSHOT} + echo "version=$VERSION" >> $GITHUB_OUTPUT + echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT + echo "PR version: $VERSION" + echo "PR Docker version: $DOCKER_VERSION" + - name: Build PR version working-directory: pr-code run: | + echo "Building PR version..." ./mvnw --no-transfer-progress clean package -DskipTests - docker build -t wrongsecrets-pr . + echo "PR JAR built successfully" + docker build --build-arg argBasedVersion="${{ steps.extract-pr-version.outputs.docker_version }}" -t wrongsecrets-pr . + echo "PR Docker image built successfully" - name: Set up JDK 23 for main uses: actions/setup-java@v4 @@ -162,11 +188,25 @@ jobs: distribution: "oracle" cache: "maven" + - name: Extract main version + id: extract-main-version + working-directory: main-code + run: | + VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + DOCKER_VERSION=${VERSION%-SNAPSHOT} + echo "version=$VERSION" >> $GITHUB_OUTPUT + echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT + echo "Main version: $VERSION" + echo "Main Docker version: $DOCKER_VERSION" + - name: Build main version working-directory: main-code run: | + echo "Building main version..." ./mvnw --no-transfer-progress clean package -DskipTests - docker build -t wrongsecrets-main . + echo "Main JAR built successfully" + docker build --build-arg argBasedVersion="${{ steps.extract-main-version.outputs.docker_version }}" -t wrongsecrets-main . + echo "Main Docker image built successfully" # Alternative approach: Pull the PR image from registry # - name: Log in to GitHub Container Registry diff --git a/.github/workflows/version-sync-check.yml b/.github/workflows/version-sync-check.yml new file mode 100644 index 000000000..ab2bec66c --- /dev/null +++ b/.github/workflows/version-sync-check.yml @@ -0,0 +1,60 @@ +name: Version Sync Check + +on: + pull_request: + paths: + - 'pom.xml' + - 'Dockerfile' + - 'Dockerfile.web' + push: + branches: [master, main] + +jobs: + version-check: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up JDK 23 + uses: actions/setup-java@v4 + with: + java-version: "23" + distribution: "oracle" + cache: "maven" + + - name: Validate version consistency + run: | + chmod +x ./scripts/validate-versions.sh + ./scripts/validate-versions.sh + + - name: Comment on PR if versions are out of sync + if: failure() && github.event_name == 'pull_request' + uses: actions/github-script@v7 + with: + script: | + const comment = `๐Ÿšจ **Version Sync Issue Detected** + + The versions in your Dockerfiles don't match the version in \`pom.xml\`. + + **๐Ÿ”ง To fix this automatically:** + \`\`\`bash + ./scripts/sync-versions.sh + git add Dockerfile Dockerfile.web + git commit -m "Sync versions with pom.xml" + \`\`\` + + **๐Ÿ“‹ Current status:** + - The \`validate-versions.sh\` script found mismatched versions + - Please ensure all Docker build arguments match the Maven project version + - This helps maintain consistency across all deployment methods + + --- + Automated version check by GitHub Actions`; + + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); diff --git a/.github/workflows/visual-diff.yml b/.github/workflows/visual-diff.yml index e0d3ffd58..b89d95509 100644 --- a/.github/workflows/visual-diff.yml +++ b/.github/workflows/visual-diff.yml @@ -28,13 +28,23 @@ jobs: distribution: "oracle" cache: "maven" + - name: Extract PR version + id: extract-pr-version + working-directory: pr-code + run: | + VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + DOCKER_VERSION=${VERSION%-SNAPSHOT} + echo "version=$VERSION" >> $GITHUB_OUTPUT + echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT + echo "PR version: $VERSION" + - name: Build PR version working-directory: pr-code run: | echo "Building PR version..." ./mvnw --no-transfer-progress clean package -DskipTests echo "PR JAR built successfully" - docker build -t wrongsecrets-pr . + docker build --build-arg argBasedVersion="${{ steps.extract-pr-version.outputs.docker_version }}" -t wrongsecrets-pr . echo "PR Docker image built successfully" - name: Set up JDK 23 for main build @@ -44,13 +54,23 @@ jobs: distribution: "oracle" cache: "maven" + - name: Extract main version + id: extract-main-version + working-directory: main-code + run: | + VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + DOCKER_VERSION=${VERSION%-SNAPSHOT} + echo "version=$VERSION" >> $GITHUB_OUTPUT + echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT + echo "Main version: $VERSION" + - name: Build main version working-directory: main-code run: | echo "Building main version..." ./mvnw --no-transfer-progress clean package -DskipTests echo "Main JAR built successfully" - docker build -t wrongsecrets-main . + docker build --build-arg argBasedVersion="${{ steps.extract-main-version.outputs.docker_version }}" -t wrongsecrets-main . echo "Main Docker image built successfully" docker build -t wrongsecrets-main . diff --git a/Dockerfile b/Dockerfile index cfc8928bd..174c0add3 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,7 +1,7 @@ FROM bellsoft/liberica-openjre-debian:23.0.2-9-cds AS builder WORKDIR /builder -ARG argBasedVersion="1.12.1" +ARG argBasedVersion="1.12.3B2" COPY --chown=wrongsecrets target/wrongsecrets-${argBasedVersion}-SNAPSHOT.jar application.jar RUN java -Djarmode=tools -jar application.jar extract --layers --destination extracted diff --git a/Dockerfile.web b/Dockerfile.web index 56afa863f..d400954bf 100644 --- a/Dockerfile.web +++ b/Dockerfile.web @@ -1,5 +1,5 @@ -FROM jeroenwillemsen/wrongsecrets:1.12.2A-no-vault -ARG argBasedVersion="1.12.2A-no-vault" +FROM jeroenwillemsen/wrongsecrets:1.12.3B2-no-vault +ARG argBasedVersion="1.12.3B2-no-vault" ARG CANARY_URLS="http://canarytokens.com/terms/about/s7cfbdakys13246ewd8ivuvku/post.jsp,http://canarytokens.com/terms/about/y0all60b627gzp19ahqh7rl6j/post.jsp" ARG CTF_ENABLED=false ARG HINTS_ENABLED=true diff --git a/docs/VERSION_MANAGEMENT.md b/docs/VERSION_MANAGEMENT.md new file mode 100644 index 000000000..f15ca9a4c --- /dev/null +++ b/docs/VERSION_MANAGEMENT.md @@ -0,0 +1,142 @@ +# Version Management Guide + +This document explains how version synchronization works across the WrongSecrets project. + +## Overview + +The project maintains version consistency between: +- `pom.xml` (Maven project version) +- `Dockerfile` (Docker build argument) +- `Dockerfile.web` (Docker build argument and base image) + +## Version Schema + +``` +pom.xml version: 1.12.3B2-SNAPSHOT +Dockerfile version: 1.12.3B2 +Dockerfile.web version: 1.12.3B2-no-vault +``` + +## Automated Solutions + +### 1. GitHub Actions Integration + +All build workflows now automatically extract the version from `pom.xml`: + +```yaml +- name: Extract version from pom.xml + id: extract-version + run: | + VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + DOCKER_VERSION=${VERSION%-SNAPSHOT} + echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT + +- name: Build Docker image + run: | + docker build --build-arg argBasedVersion="${{ steps.extract-version.outputs.docker_version }}" -t image . +``` + +### 2. Version Sync Scripts + +#### Validate Versions +```bash +./scripts/validate-versions.sh +``` +Checks if all versions are consistent and reports mismatches. + +#### Auto-Sync Versions +```bash +./scripts/sync-versions.sh +``` +Automatically updates Dockerfiles to match `pom.xml` version. + +#### Build with Version Sync +```bash +./scripts/build-with-version-sync.sh +``` +Builds both Docker images with correct versions from `pom.xml`. + +### 3. CI/CD Integration + +The `version-sync-check.yml` workflow: +- โœ… Runs on PR/push when version files change +- โœ… Validates version consistency +- โœ… Comments on PRs with fix instructions if mismatched +- โœ… Prevents version drift + +## Manual Process + +### When Updating Versions + +1. **Update pom.xml version**: + ```xml + 1.13.0-SNAPSHOT + ``` + +2. **Run sync script**: + ```bash + ./scripts/sync-versions.sh + ``` + +3. **Verify changes**: + ```bash + ./scripts/validate-versions.sh + ``` + +4. **Commit all changes**: + ```bash + git add pom.xml Dockerfile Dockerfile.web + git commit -m "Bump version to 1.13.0" + ``` + +## Workflow Integration + +### All Build Workflows Include: + +1. **Version Extraction**: Gets version from `pom.xml` +2. **Dynamic Build Args**: Passes version to Docker build +3. **Validation**: Ensures JAR file matches expected name +4. **Logging**: Shows which versions are being used + +### Benefits: + +- โœ… **Single Source of Truth**: `pom.xml` is the authoritative version +- โœ… **No Manual Updates**: Dockerfiles auto-sync with Maven version +- โœ… **CI Validation**: Catches version mismatches early +- โœ… **Consistent Builds**: Same version used across all environments + +## Troubleshooting + +### Common Issues: + +1. **JAR Not Found**: Version mismatch between build arg and actual JAR name + - **Solution**: Run `./scripts/sync-versions.sh` + +2. **Docker Build Fails**: Hard-coded version in Dockerfile + - **Solution**: Use `--build-arg argBasedVersion=...` + +3. **CI Version Mismatch**: Manual updates to Dockerfiles + - **Solution**: Let CI extract from `pom.xml` dynamically + +### Debug Commands: + +```bash +# Check current versions +mvn help:evaluate -Dexpression=project.version -q -DforceStdout +grep "argBasedVersion" Dockerfile Dockerfile.web + +# Test build with current version +VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +DOCKER_VERSION=${VERSION%-SNAPSHOT} +docker build --build-arg argBasedVersion="$DOCKER_VERSION" . +``` + +## Best Practices + +1. **Always use scripts** for version updates +2. **Never hard-code versions** in CI workflows +3. **Run validation** before committing changes +4. **Update pom.xml first**, then sync other files +5. **Test builds locally** before pushing + +This system ensures version consistency and eliminates manual synchronization errors! diff --git a/scripts/build-with-version-sync.sh b/scripts/build-with-version-sync.sh new file mode 100755 index 000000000..58100a249 --- /dev/null +++ b/scripts/build-with-version-sync.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# Script to extract version from pom.xml and build Docker images with correct version + +set -e + +# Extract version from pom.xml +VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +echo "Detected version from pom.xml: $VERSION" + +# Remove -SNAPSHOT suffix for Docker tags if present +DOCKER_VERSION=${VERSION%-SNAPSHOT} +echo "Docker version will be: $DOCKER_VERSION" + +# Build the application first +echo "Building application..." +./mvnw clean package -DskipTests + +# Build main Docker image +echo "Building main Docker image..." +docker build --build-arg argBasedVersion="$DOCKER_VERSION" -t "wrongsecrets:$DOCKER_VERSION" . + +# Build web Docker image +echo "Building web Docker image..." +docker build --build-arg argBasedVersion="$DOCKER_VERSION-no-vault" -f Dockerfile.web -t "wrongsecrets:$DOCKER_VERSION-no-vault" . + +echo "Build completed successfully!" +echo "Images built:" +echo " - wrongsecrets:$DOCKER_VERSION" +echo " - wrongsecrets:$DOCKER_VERSION-no-vault" diff --git a/scripts/sync-versions.sh b/scripts/sync-versions.sh new file mode 100755 index 000000000..48cf91589 --- /dev/null +++ b/scripts/sync-versions.sh @@ -0,0 +1,36 @@ +#!/bin/bash +# Script to automatically sync versions across all files + +set -e + +echo "๐Ÿ”„ Syncing versions across all files..." + +# Extract version from pom.xml +POM_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +BASE_VERSION=${POM_VERSION%-SNAPSHOT} +WEB_VERSION="$BASE_VERSION-no-vault" + +echo "๐Ÿ“„ Source version (pom.xml): $POM_VERSION" +echo "๐Ÿณ Target Docker version: $BASE_VERSION" +echo "๐ŸŒ Target web version: $WEB_VERSION" + +# Update Dockerfile +echo "๐Ÿ”ง Updating Dockerfile..." +sed -i.bak "s/^ARG argBasedVersion=.*/ARG argBasedVersion=\"$BASE_VERSION\"/" Dockerfile +rm -f Dockerfile.bak + +# Update Dockerfile.web +echo "๐Ÿ”ง Updating Dockerfile.web..." +sed -i.bak "s/^FROM jeroenwillemsen\/wrongsecrets:.*/FROM jeroenwillemsen\/wrongsecrets:$WEB_VERSION/" Dockerfile.web +sed -i.bak "s/^ARG argBasedVersion=.*/ARG argBasedVersion=\"$WEB_VERSION\"/" Dockerfile.web +rm -f Dockerfile.web.bak + +# Verify changes +echo "โœ… Version sync completed!" +echo "" +echo "๐Ÿ“‹ Updated versions:" +echo " - Dockerfile: $(grep "^ARG argBasedVersion=" Dockerfile | cut -d'"' -f2)" +echo " - Dockerfile.web: $(grep "^ARG argBasedVersion=" Dockerfile.web | cut -d'"' -f2)" +echo " - Dockerfile.web base image: $(grep "^FROM jeroenwillemsen/wrongsecrets:" Dockerfile.web | cut -d':' -f2)" +echo "" +echo "๐Ÿ” Run './scripts/validate-versions.sh' to verify all versions are in sync." diff --git a/scripts/validate-versions.sh b/scripts/validate-versions.sh new file mode 100755 index 000000000..279db06a0 --- /dev/null +++ b/scripts/validate-versions.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# Script to validate that all version references are in sync + +set -e + +echo "๐Ÿ” Checking version consistency across files..." + +# Extract version from pom.xml +POM_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +echo "๐Ÿ“„ pom.xml version: $POM_VERSION" + +# Extract base version (remove -SNAPSHOT) +BASE_VERSION=${POM_VERSION%-SNAPSHOT} +echo "๐Ÿณ Expected Docker base version: $BASE_VERSION" + +# Check Dockerfile +DOCKERFILE_VERSION=$(grep "^ARG argBasedVersion=" Dockerfile | cut -d'"' -f2) +echo "๐Ÿณ Dockerfile argBasedVersion: $DOCKERFILE_VERSION" + +# Check Dockerfile.web +DOCKERFILE_WEB_VERSION=$(grep "^ARG argBasedVersion=" Dockerfile.web | cut -d'"' -f2) +echo "๐ŸŒ Dockerfile.web argBasedVersion: $DOCKERFILE_WEB_VERSION" + +# Validation +ERRORS=0 + +if [ "$DOCKERFILE_VERSION" != "$BASE_VERSION" ]; then + echo "โŒ ERROR: Dockerfile version ($DOCKERFILE_VERSION) doesn't match pom.xml base version ($BASE_VERSION)" + ERRORS=$((ERRORS + 1)) +fi + +# For Dockerfile.web, check if it follows the pattern BASE_VERSION-no-vault +EXPECTED_WEB_VERSION="$BASE_VERSION-no-vault" +if [ "$DOCKERFILE_WEB_VERSION" != "$EXPECTED_WEB_VERSION" ]; then + echo "โŒ ERROR: Dockerfile.web version ($DOCKERFILE_WEB_VERSION) doesn't match expected pattern ($EXPECTED_WEB_VERSION)" + ERRORS=$((ERRORS + 1)) +fi + +# Check base image in Dockerfile.web +BASE_IMAGE_VERSION=$(grep "^FROM jeroenwillemsen/wrongsecrets:" Dockerfile.web | cut -d':' -f2) +echo "๐Ÿ—๏ธ Dockerfile.web base image version: $BASE_IMAGE_VERSION" + +if [ "$BASE_IMAGE_VERSION" != "$EXPECTED_WEB_VERSION" ]; then + echo "โŒ ERROR: Dockerfile.web base image version ($BASE_IMAGE_VERSION) doesn't match argBasedVersion ($DOCKERFILE_WEB_VERSION)" + ERRORS=$((ERRORS + 1)) +fi + +# Summary +if [ $ERRORS -eq 0 ]; then + echo "โœ… All versions are in sync!" + exit 0 +else + echo "โŒ Found $ERRORS version mismatches. Please fix them to ensure consistency." + echo "" + echo "๐Ÿ”ง To fix automatically, run:" + echo " ./scripts/sync-versions.sh" + exit 1 +fi From 7e7a811544ab175e3b8e720ec04c293828bf8f39 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 07:32:43 +0200 Subject: [PATCH 07/16] retry with ai --- .github/workflows/build-preview.yml | 4 ++- .github/workflows/pr-preview.yml | 41 ++++++++++++++++++++++++----- .github/workflows/visual-diff.yml | 8 ++++-- scripts/build-with-version-sync.sh | 2 +- scripts/sync-versions.sh | 2 +- scripts/validate-versions.sh | 2 +- 6 files changed, 47 insertions(+), 12 deletions(-) diff --git a/.github/workflows/build-preview.yml b/.github/workflows/build-preview.yml index 2e78b32a3..4643f45cf 100644 --- a/.github/workflows/build-preview.yml +++ b/.github/workflows/build-preview.yml @@ -25,7 +25,9 @@ jobs: - name: Extract version from pom.xml id: extract-version run: | - VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Extracting version from pom.xml..." + chmod +x ./mvnw + VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) DOCKER_VERSION=${VERSION%-SNAPSHOT} echo "version=$VERSION" >> $GITHUB_OUTPUT echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index ca4072d25..57afb3b41 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -32,19 +32,39 @@ jobs: distribution: "oracle" cache: "maven" - - name: Build application - run: ./mvnw --no-transfer-progress clean package -DskipTests - - name: Extract version from pom.xml id: extract-version run: | - VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Extracting version from pom.xml..." + chmod +x ./mvnw + VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) DOCKER_VERSION=${VERSION%-SNAPSHOT} echo "version=$VERSION" >> $GITHUB_OUTPUT echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT echo "Detected version: $VERSION" echo "Docker version: $DOCKER_VERSION" + - name: Build application + run: ./mvnw --no-transfer-progress clean package -DskipTests + + - name: Verify JAR file was created + run: | + echo "Checking target directory..." + ls -la target/ + echo "Looking for JAR files..." + find target/ -name "*.jar" -type f + echo "Verifying specific JAR exists..." + JAR_FILE="target/wrongsecrets-${{ steps.extract-version.outputs.version }}.jar" + if [ -f "$JAR_FILE" ]; then + echo "โœ… JAR file found: $JAR_FILE" + ls -la "$JAR_FILE" + else + echo "โŒ Expected JAR file not found: $JAR_FILE" + echo "Available JAR files:" + find target/ -name "*.jar" -type f || echo "No JAR files found" + exit 1 + fi + - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 @@ -77,6 +97,11 @@ jobs: cache-from: type=gha cache-to: type=gha,mode=max + - name: Verify Docker image was built + run: | + echo "Verifying Docker image was built successfully..." + docker images | grep wrongsecrets-pr || echo "No wrongsecrets-pr images found" + # Comment out Render deployment for now # - name: Deploy to Render (Preview) # env: @@ -165,7 +190,9 @@ jobs: id: extract-pr-version working-directory: pr-code run: | - VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Extracting PR version from pom.xml..." + chmod +x ./mvnw + VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) DOCKER_VERSION=${VERSION%-SNAPSHOT} echo "version=$VERSION" >> $GITHUB_OUTPUT echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT @@ -192,7 +219,9 @@ jobs: id: extract-main-version working-directory: main-code run: | - VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Extracting main version from pom.xml..." + chmod +x ./mvnw + VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) DOCKER_VERSION=${VERSION%-SNAPSHOT} echo "version=$VERSION" >> $GITHUB_OUTPUT echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT diff --git a/.github/workflows/visual-diff.yml b/.github/workflows/visual-diff.yml index b89d95509..99fd6f21e 100644 --- a/.github/workflows/visual-diff.yml +++ b/.github/workflows/visual-diff.yml @@ -32,7 +32,9 @@ jobs: id: extract-pr-version working-directory: pr-code run: | - VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Extracting PR version..." + chmod +x ./mvnw + VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) DOCKER_VERSION=${VERSION%-SNAPSHOT} echo "version=$VERSION" >> $GITHUB_OUTPUT echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT @@ -58,7 +60,9 @@ jobs: id: extract-main-version working-directory: main-code run: | - VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) + echo "Extracting main version..." + chmod +x ./mvnw + VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) DOCKER_VERSION=${VERSION%-SNAPSHOT} echo "version=$VERSION" >> $GITHUB_OUTPUT echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT diff --git a/scripts/build-with-version-sync.sh b/scripts/build-with-version-sync.sh index 58100a249..c29f2ed48 100755 --- a/scripts/build-with-version-sync.sh +++ b/scripts/build-with-version-sync.sh @@ -4,7 +4,7 @@ set -e # Extract version from pom.xml -VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) echo "Detected version from pom.xml: $VERSION" # Remove -SNAPSHOT suffix for Docker tags if present diff --git a/scripts/sync-versions.sh b/scripts/sync-versions.sh index 48cf91589..09dacfe31 100755 --- a/scripts/sync-versions.sh +++ b/scripts/sync-versions.sh @@ -6,7 +6,7 @@ set -e echo "๐Ÿ”„ Syncing versions across all files..." # Extract version from pom.xml -POM_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +POM_VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) BASE_VERSION=${POM_VERSION%-SNAPSHOT} WEB_VERSION="$BASE_VERSION-no-vault" diff --git a/scripts/validate-versions.sh b/scripts/validate-versions.sh index 279db06a0..7f8a52b5a 100755 --- a/scripts/validate-versions.sh +++ b/scripts/validate-versions.sh @@ -6,7 +6,7 @@ set -e echo "๐Ÿ” Checking version consistency across files..." # Extract version from pom.xml -POM_VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout) +POM_VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) echo "๐Ÿ“„ pom.xml version: $POM_VERSION" # Extract base version (remove -SNAPSHOT) From 33cd162568408df12ebd2a3741ba29822b5519d0 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 07:35:55 +0200 Subject: [PATCH 08/16] quick fixes --- .github/workflows/pr-preview.yml | 28 +++++++++++++++++++++++-- test-workflow.sh | 36 ++++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 test-workflow.sh diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 57afb3b41..99853b3a5 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -102,6 +102,21 @@ jobs: echo "Verifying Docker image was built successfully..." docker images | grep wrongsecrets-pr || echo "No wrongsecrets-pr images found" + - name: Save Docker image as artifact + run: | + echo "Saving Docker image as tar artifact..." + IMAGE_TAG="${{ steps.meta.outputs.tags }}" | head -n1 + docker save "$IMAGE_TAG" -o wrongsecrets-preview.tar + echo "Docker image saved to wrongsecrets-preview.tar" + ls -lh wrongsecrets-preview.tar + + - name: Upload Docker image artifact + uses: actions/upload-artifact@v4 + with: + name: wrongsecrets-preview-pr-${{ github.event.number }} + path: wrongsecrets-preview.tar + retention-days: 30 + # Comment out Render deployment for now # - name: Deploy to Render (Preview) # env: @@ -130,6 +145,7 @@ jobs: script: | const prNumber = context.issue.number; const imageTag = `${{ steps.meta.outputs.tags }}`.split('\n')[0]; + const runId = context.runId; const comment = `๐Ÿ”จ **Preview Build Complete!** @@ -137,9 +153,17 @@ jobs: **๐Ÿณ Docker Image:** \`${imageTag}\` - **๐Ÿš€ To test locally:** + **๏ฟฝ Download & Test Locally:** + 1. [๐Ÿ“ Download Docker Image Artifact](https://github.com/${{ github.repository }}/actions/runs/${runId}) (look for \`wrongsecrets-preview-pr-${prNumber}\`) + 2. Load and run the image: + \`\`\`bash + # Download the artifact, extract it, then: + docker load < wrongsecrets-preview.tar + docker run -p 8080:8080 wrongsecrets-preview + \`\`\` + + **๐Ÿš€ Alternative - Pull from Registry:** \`\`\`bash - # Pull and run the preview image docker pull ${imageTag} docker run -p 8080:8080 ${imageTag} \`\`\` diff --git a/test-workflow.sh b/test-workflow.sh new file mode 100644 index 000000000..1aeb59419 --- /dev/null +++ b/test-workflow.sh @@ -0,0 +1,36 @@ +#!/bin/bash +set -e + +echo "๐Ÿ”ง Testing workflow commands locally..." + +# Make sure Maven wrapper is executable +chmod +x ./mvnw + +# Extract version +echo "๐Ÿ“ฆ Extracting version from pom.xml..." +VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) +echo "Version: $VERSION" + +# Verify the version format +if [[ -z "$VERSION" ]]; then + echo "โŒ Version extraction failed" + exit 1 +fi + +echo "โœ… Version extracted successfully: $VERSION" + +# Test compilation (without running full package to save time) +echo "๐Ÿ—๏ธ Testing compilation..." +./mvnw compile -q + +echo "โœ… Compilation successful" + +# Check if target directory exists +if [[ -d "target" ]]; then + echo "โœ… Target directory exists" +else + echo "โŒ Target directory missing" + exit 1 +fi + +echo "๐ŸŽ‰ All workflow prerequisite tests passed!" From 0cac90902df89b38f0b9276d2434e548fec83dcb Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 07:39:49 +0200 Subject: [PATCH 09/16] remove cleanup --- .github/workflows/pr-preview.yml | 88 ++++++++++++++++---------------- 1 file changed, 44 insertions(+), 44 deletions(-) diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 99853b3a5..61433d7e4 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -377,33 +377,33 @@ jobs: body: comment }); - cleanup-preview: - runs-on: ubuntu-latest - if: github.event.action == 'closed' - steps: - - name: Log in to GitHub Container Registry - uses: docker/login-action@v3 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - name: Delete PR container images - run: | - # Delete container images for this PR - PR_NUMBER=${{ github.event.number }} - REPO_LOWER=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]') - - # Get all tags for this PR and delete them - for tag in $(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ - "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions" | \ - jq -r --arg pr "$PR_NUMBER" '.[] | select(.metadata.container.tags[]? | contains($pr)) | .id'); do - - echo "Deleting container version: $tag" - curl -X DELETE \ - -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ - "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions/$tag" - done +# cleanup-preview: +# runs-on: ubuntu-latest +# if: github.event.action == 'closed' +# steps: +# - name: Log in to GitHub Container Registry +# uses: docker/login-action@v3 +# with: +# registry: ghcr.io +# username: ${{ github.actor }} +# password: ${{ secrets.GITHUB_TOKEN }} + +# - name: Delete PR container images +# run: | +# # Delete container images for this PR +# PR_NUMBER=${{ github.event.number }} +# REPO_LOWER=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]') + +# # Get all tags for this PR and delete them +# for tag in $(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ +# "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions" | \ +# jq -r --arg pr "$PR_NUMBER" '.[] | select(.metadata.container.tags[]? | contains($pr)) | .id'); do + +# echo "Deleting container version: $tag" +# curl -X DELETE \ +# -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ +# "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions/$tag" +# done # Comment out Render cleanup for now # - name: Cleanup Render service @@ -420,24 +420,24 @@ jobs: # -H "Authorization: Bearer $RENDER_API_KEY" # fi - - name: Comment PR closure - uses: actions/github-script@v7 - with: - script: | - const comment = `๐Ÿงน **Preview Cleanup Complete** + # - name: Comment PR closure + # uses: actions/github-script@v7 + # with: + # script: | + # const comment = `๐Ÿงน **Preview Cleanup Complete** - PR preview resources have been cleaned up: - - โœ… Container images deleted from GitHub Container Registry - - โœ… Artifacts will expire automatically in 30 days + # PR preview resources have been cleaned up: + # - โœ… Container images deleted from GitHub Container Registry + # - โœ… Artifacts will expire automatically in 30 days - Thanks for contributing to WrongSecrets! ๐ŸŽ‰ + # Thanks for contributing to WrongSecrets! ๐ŸŽ‰ - --- - Cleanup completed by GitHub Actions`; + # --- + # Cleanup completed by GitHub Actions`; - github.rest.issues.createComment({ - issue_number: context.issue.number, - owner: context.repo.owner, - repo: context.repo.repo, - body: comment - }); + # github.rest.issues.createComment({ + # issue_number: context.issue.number, + # owner: context.repo.owner, + # repo: context.repo.repo, + # body: comment + # }); From 7b1a99141f209d439b93efc1787a5986d995b342 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 07:49:54 +0200 Subject: [PATCH 10/16] now with yml fixes --- .github/workflows/pr-preview.yml | 99 +++++++++++++++++--------------- 1 file changed, 53 insertions(+), 46 deletions(-) diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 61433d7e4..9723a3fcc 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -105,7 +105,14 @@ jobs: - name: Save Docker image as artifact run: | echo "Saving Docker image as tar artifact..." - IMAGE_TAG="${{ steps.meta.outputs.tags }}" | head -n1 + IMAGE_TAG=$(echo "${{ steps.meta.outputs.tags }}" | head -n1) + echo "Using image tag: $IMAGE_TAG" + + # Pull the image locally if it's not available + echo "Pulling image to ensure it's available locally..." + docker pull "$IMAGE_TAG" + + # Save the image as tar docker save "$IMAGE_TAG" -o wrongsecrets-preview.tar echo "Docker image saved to wrongsecrets-preview.tar" ls -lh wrongsecrets-preview.tar @@ -153,7 +160,7 @@ jobs: **๐Ÿณ Docker Image:** \`${imageTag}\` - **๏ฟฝ Download & Test Locally:** + **๐Ÿ“ฆ Download & Test Locally:** 1. [๐Ÿ“ Download Docker Image Artifact](https://github.com/${{ github.repository }}/actions/runs/${runId}) (look for \`wrongsecrets-preview-pr-${prNumber}\`) 2. Load and run the image: \`\`\`bash @@ -377,33 +384,33 @@ jobs: body: comment }); -# cleanup-preview: -# runs-on: ubuntu-latest -# if: github.event.action == 'closed' -# steps: -# - name: Log in to GitHub Container Registry -# uses: docker/login-action@v3 -# with: -# registry: ghcr.io -# username: ${{ github.actor }} -# password: ${{ secrets.GITHUB_TOKEN }} - -# - name: Delete PR container images -# run: | -# # Delete container images for this PR -# PR_NUMBER=${{ github.event.number }} -# REPO_LOWER=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]') - -# # Get all tags for this PR and delete them -# for tag in $(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ -# "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions" | \ -# jq -r --arg pr "$PR_NUMBER" '.[] | select(.metadata.container.tags[]? | contains($pr)) | .id'); do - -# echo "Deleting container version: $tag" -# curl -X DELETE \ -# -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ -# "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions/$tag" -# done + cleanup-preview: + runs-on: ubuntu-latest + if: github.event.action == 'closed' + steps: + - name: Log in to GitHub Container Registry + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Delete PR container images + run: | + # Delete container images for this PR + PR_NUMBER=${{ github.event.number }} + REPO_LOWER=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]') + + # Get all tags for this PR and delete them + for tag in $(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ + "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions" | \ + jq -r --arg pr "$PR_NUMBER" '.[] | select(.metadata.container.tags[]? | contains($pr)) | .id'); do + + echo "Deleting container version: $tag" + curl -X DELETE \ + -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ + "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions/$tag" + done # Comment out Render cleanup for now # - name: Cleanup Render service @@ -420,24 +427,24 @@ jobs: # -H "Authorization: Bearer $RENDER_API_KEY" # fi - # - name: Comment PR closure - # uses: actions/github-script@v7 - # with: - # script: | - # const comment = `๐Ÿงน **Preview Cleanup Complete** + - name: Comment PR closure + uses: actions/github-script@v7 + with: + script: | + const comment = `๐Ÿงน **Preview Cleanup Complete** - # PR preview resources have been cleaned up: - # - โœ… Container images deleted from GitHub Container Registry - # - โœ… Artifacts will expire automatically in 30 days + PR preview resources have been cleaned up: + - โœ… Container images deleted from GitHub Container Registry + - โœ… Artifacts will expire automatically in 30 days - # Thanks for contributing to WrongSecrets! ๐ŸŽ‰ + Thanks for contributing to WrongSecrets! ๐ŸŽ‰ - # --- - # Cleanup completed by GitHub Actions`; + --- + Cleanup completed by GitHub Actions`; - # github.rest.issues.createComment({ - # issue_number: context.issue.number, - # owner: context.repo.owner, - # repo: context.repo.repo, - # body: comment - # }); + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: comment + }); From 69d57061f9d3c3950337ee02eba3910545fc579a Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 08:03:25 +0200 Subject: [PATCH 11/16] feedback enhancements --- .github/workflows/build-preview.yml | 42 ++++++++++++++++++++++++----- .github/workflows/pr-preview.yml | 33 +++++++++++++++++++---- 2 files changed, 64 insertions(+), 11 deletions(-) diff --git a/.github/workflows/build-preview.yml b/.github/workflows/build-preview.yml index 4643f45cf..95ee6cd2e 100644 --- a/.github/workflows/build-preview.yml +++ b/.github/workflows/build-preview.yml @@ -11,6 +11,9 @@ on: jobs: build-preview: runs-on: ubuntu-latest + # Temporarily disabled to avoid duplicate comments with pr-preview.yml + # Both workflows provide similar functionality, pr-preview.yml is more comprehensive + if: false steps: - name: Checkout code uses: actions/checkout@v4 @@ -60,27 +63,54 @@ jobs: uses: actions/github-script@v7 with: script: | + const prNumber = context.issue.number; + const runId = context.runId; + const comment = `๐Ÿ”จ **Preview Build Complete!** - Your changes have been built successfully. To test locally: + Your changes have been built successfully. + **๐Ÿ“ฆ Download & Test Locally:** + 1. [๐Ÿ“ Download Docker Image Artifact](https://github.com/${{ github.repository }}/actions/runs/${runId}) (look for \`wrongsecrets-preview-${prNumber}\`) + 2. Load and run the image: \`\`\`bash - # Download the artifact and load the Docker image + # Download the artifact, extract it, then: docker load < wrongsecrets-preview.tar docker run -p 8080:8080 wrongsecrets-preview \`\`\` Then visit: http://localhost:8080 - **Changed Files in this PR:** - - Templates: ${{ github.event.pull_request.changed_files }} + **๐Ÿ“ Changed Files in this PR:**`; + + // Get the list of changed files + const { data: files } = await github.rest.pulls.listFiles({ + owner: context.repo.owner, + repo: context.repo.repo, + pull_number: prNumber + }); + + const templateFiles = files.filter(file => + file.filename.includes('templates/') || + file.filename.includes('static/') || + file.filename.includes('explanations/') + ); + + let filesList = ''; + if (templateFiles.length > 0) { + filesList = templateFiles.map(file => `- \`${file.filename}\``).join('\\n '); + } else { + filesList = '- No template/static files changed'; + } + + const fullComment = comment + '\\n ' + filesList + ` --- Build completed by GitHub Actions`; github.rest.issues.createComment({ - issue_number: context.issue.number, + issue_number: prNumber, owner: context.repo.owner, repo: context.repo.repo, - body: comment + body: fullComment }); diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 9723a3fcc..c248bd686 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -177,10 +177,33 @@ jobs: Then visit: http://localhost:8080 - **๐Ÿ“ Changes in this PR:** - - Templates and static content updates - - Challenge explanations - - Application logic changes + **๐Ÿ“ Changes in this PR:**`; + + // Get the list of changed files + const { data: files } = await github.rest.pulls.listFiles({ + owner: context.repo.owner, + repo: context.repo.repo, + pull_number: prNumber + }); + + const relevantFiles = files.filter(file => + file.filename.includes('templates/') || + file.filename.includes('static/') || + file.filename.includes('explanations/') || + file.filename.includes('src/main/java/') + ); + + let filesList = ''; + if (relevantFiles.length > 0) { + filesList = relevantFiles.slice(0, 10).map(file => `- \`${file.filename}\``).join('\\n '); + if (relevantFiles.length > 10) { + filesList += `\\n - ... and ${relevantFiles.length - 10} more files`; + } + } else { + filesList = '- No relevant files changed'; + } + + const finalComment = comment + '\\n ' + filesList + ` Visual diff screenshots will be available shortly... @@ -191,7 +214,7 @@ jobs: issue_number: prNumber, owner: context.repo.owner, repo: context.repo.repo, - body: comment + body: finalComment }); visual-diff: From 28f0efc9fa642ae4e91b14184ab72b7e8a333e4b Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 08:20:45 +0200 Subject: [PATCH 12/16] additional fixes to make playwrite work --- .github/workflows/build-preview.yml | 4 +- .github/workflows/pr-preview.yml | 57 ++++++++++++++++++++-------- .github/workflows/visual-diff.yml | 59 +++++++++++++++++++---------- 3 files changed, 83 insertions(+), 37 deletions(-) diff --git a/.github/workflows/build-preview.yml b/.github/workflows/build-preview.yml index 95ee6cd2e..17f76f8e6 100644 --- a/.github/workflows/build-preview.yml +++ b/.github/workflows/build-preview.yml @@ -98,12 +98,12 @@ jobs: let filesList = ''; if (templateFiles.length > 0) { - filesList = templateFiles.map(file => `- \`${file.filename}\``).join('\\n '); + filesList = templateFiles.map(file => `- \`${file.filename}\``).join('\n '); } else { filesList = '- No template/static files changed'; } - const fullComment = comment + '\\n ' + filesList + ` + const fullComment = comment + '\n ' + filesList + ` --- Build completed by GitHub Actions`; diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index c248bd686..82667dfe3 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -108,9 +108,20 @@ jobs: IMAGE_TAG=$(echo "${{ steps.meta.outputs.tags }}" | head -n1) echo "Using image tag: $IMAGE_TAG" - # Pull the image locally if it's not available - echo "Pulling image to ensure it's available locally..." - docker pull "$IMAGE_TAG" + # Check if image is available locally first + if docker images --format "table {{.Repository}}:{{.Tag}}" | grep -q "$IMAGE_TAG"; then + echo "Image found locally, using local image" + else + echo "Image not found locally, attempting to pull..." + if docker pull "$IMAGE_TAG"; then + echo "Successfully pulled image" + else + echo "Failed to pull image, but this might be expected immediately after push" + echo "Waiting 10 seconds and trying again..." + sleep 10 + docker pull "$IMAGE_TAG" || echo "Still failed to pull, continuing with local build" + fi + fi # Save the image as tar docker save "$IMAGE_TAG" -o wrongsecrets-preview.tar @@ -195,15 +206,15 @@ jobs: let filesList = ''; if (relevantFiles.length > 0) { - filesList = relevantFiles.slice(0, 10).map(file => `- \`${file.filename}\``).join('\\n '); + filesList = relevantFiles.slice(0, 10).map(file => `- \`${file.filename}\``).join('\n '); if (relevantFiles.length > 10) { - filesList += `\\n - ... and ${relevantFiles.length - 10} more files`; + filesList += `\n - ... and ${relevantFiles.length - 10} more files`; } } else { filesList = '- No relevant files changed'; } - const finalComment = comment + '\\n ' + filesList + ` + const finalComment = comment + '\n ' + filesList + ` Visual diff screenshots will be available shortly... @@ -320,48 +331,64 @@ jobs: sleep 2 done + - name: Setup Node.js + uses: actions/setup-node@v4 + with: + node-version: '18' + - name: Install Playwright run: | - npm install -g playwright@latest - playwright install chromium + npm install playwright@latest + npx playwright install --with-deps chromium - name: Take screenshots run: | mkdir -p screenshots + # Verify services are still running + echo "Verifying services are still running..." + docker ps --filter "name=pr-version" --format "table {{.Names}}\t{{.Status}}" + docker ps --filter "name=main-version" --format "table {{.Names}}\t{{.Status}}" + + # Test connectivity one more time + echo "Testing connectivity..." + curl -s -o /dev/null -w "%{http_code}" http://localhost:8080 || echo "PR version not responding" + curl -s -o /dev/null -w "%{http_code}" http://localhost:8081 || echo "Main version not responding" + node -e " const { chromium } = require('playwright'); (async () => { - const browser = await chromium.launch(); + const browser = await chromium.launch({ headless: true }); const page = await browser.newPage(); await page.setViewportSize({ width: 1280, height: 1024 }); try { // PR version screenshots console.log('Taking PR screenshots...'); - await page.goto('http://localhost:8080', { waitUntil: 'networkidle' }); + await page.goto('http://localhost:8080', { waitUntil: 'networkidle', timeout: 30000 }); await page.screenshot({ path: 'screenshots/pr-home.png', fullPage: true }); - await page.goto('http://localhost:8080/about', { waitUntil: 'networkidle' }); + await page.goto('http://localhost:8080/about', { waitUntil: 'networkidle', timeout: 30000 }); await page.screenshot({ path: 'screenshots/pr-about.png', fullPage: true }); // Try to get a challenge page - await page.goto('http://localhost:8080/challenge/1', { waitUntil: 'networkidle' }); + await page.goto('http://localhost:8080/challenge/1', { waitUntil: 'networkidle', timeout: 30000 }); await page.screenshot({ path: 'screenshots/pr-challenge.png', fullPage: true }); // Main version screenshots console.log('Taking main branch screenshots...'); - await page.goto('http://localhost:8081', { waitUntil: 'networkidle' }); + await page.goto('http://localhost:8081', { waitUntil: 'networkidle', timeout: 30000 }); await page.screenshot({ path: 'screenshots/main-home.png', fullPage: true }); - await page.goto('http://localhost:8081/about', { waitUntil: 'networkidle' }); + await page.goto('http://localhost:8081/about', { waitUntil: 'networkidle', timeout: 30000 }); await page.screenshot({ path: 'screenshots/main-about.png', fullPage: true }); - await page.goto('http://localhost:8081/challenge/1', { waitUntil: 'networkidle' }); + await page.goto('http://localhost:8081/challenge/1', { waitUntil: 'networkidle', timeout: 30000 }); await page.screenshot({ path: 'screenshots/main-challenge.png', fullPage: true }); } catch (error) { console.error('Screenshot error:', error); + process.exit(1); } finally { await browser.close(); } diff --git a/.github/workflows/visual-diff.yml b/.github/workflows/visual-diff.yml index 99fd6f21e..e78b6d426 100644 --- a/.github/workflows/visual-diff.yml +++ b/.github/workflows/visual-diff.yml @@ -84,37 +84,56 @@ jobs: docker run -d -p 8081:8080 --name main-version wrongsecrets-main sleep 30 + - name: Setup Node.js + uses: actions/setup-node@v4 + with: + node-version: '18' + - name: Install Playwright run: | - npm install -g playwright - playwright install chromium + npm install playwright@latest + npx playwright install --with-deps chromium - name: Take screenshots run: | mkdir -p screenshots - # Screenshot main pages + # Verify services are running + echo "Verifying services are running..." + docker ps --filter "name=pr-version" --format "table {{.Names}}\t{{.Status}}" + docker ps --filter "name=main-version" --format "table {{.Names}}\t{{.Status}}" + + # Screenshot main pages with error handling node -e " const { chromium } = require('playwright'); (async () => { - const browser = await chromium.launch(); + const browser = await chromium.launch({ headless: true }); const page = await browser.newPage(); - - // PR version screenshots - await page.goto('http://localhost:8080'); - await page.screenshot({ path: 'screenshots/pr-home.png', fullPage: true }); - - await page.goto('http://localhost:8080/about'); - await page.screenshot({ path: 'screenshots/pr-about.png', fullPage: true }); - - // Main version screenshots - await page.goto('http://localhost:8081'); - await page.screenshot({ path: 'screenshots/main-home.png', fullPage: true }); - - await page.goto('http://localhost:8081/about'); - await page.screenshot({ path: 'screenshots/main-about.png', fullPage: true }); - - await browser.close(); + await page.setViewportSize({ width: 1280, height: 1024 }); + + try { + // PR version screenshots + console.log('Taking PR screenshots...'); + await page.goto('http://localhost:8080', { waitUntil: 'networkidle', timeout: 30000 }); + await page.screenshot({ path: 'screenshots/pr-home.png', fullPage: true }); + + await page.goto('http://localhost:8080/about', { waitUntil: 'networkidle', timeout: 30000 }); + await page.screenshot({ path: 'screenshots/pr-about.png', fullPage: true }); + + // Main version screenshots + console.log('Taking main branch screenshots...'); + await page.goto('http://localhost:8081', { waitUntil: 'networkidle', timeout: 30000 }); + await page.screenshot({ path: 'screenshots/main-home.png', fullPage: true }); + + await page.goto('http://localhost:8081/about', { waitUntil: 'networkidle', timeout: 30000 }); + await page.screenshot({ path: 'screenshots/main-about.png', fullPage: true }); + + } catch (error) { + console.error('Screenshot error:', error); + process.exit(1); + } finally { + await browser.close(); + } })(); " From 9bfb76af9d659c400ae13e033dac9f6e5c2a3f09 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 08:31:00 +0200 Subject: [PATCH 13/16] fix: resolve visual-diff workflow issues - Remove duplicate Docker build command - Add proper container health checks and error handling - Improve service readiness verification with timeouts - Add container logs on failure for debugging --- .github/workflows/visual-diff.yml | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/.github/workflows/visual-diff.yml b/.github/workflows/visual-diff.yml index e78b6d426..105c091da 100644 --- a/.github/workflows/visual-diff.yml +++ b/.github/workflows/visual-diff.yml @@ -76,14 +76,34 @@ jobs: echo "Main JAR built successfully" docker build --build-arg argBasedVersion="${{ steps.extract-main-version.outputs.docker_version }}" -t wrongsecrets-main . echo "Main Docker image built successfully" - docker build -t wrongsecrets-main . - name: Start both versions run: | docker run -d -p 8080:8080 --name pr-version wrongsecrets-pr docker run -d -p 8081:8080 --name main-version wrongsecrets-main + + # Wait for containers to start + echo "Waiting for containers to start..." sleep 30 + # Verify containers are running + if ! docker ps --filter "name=pr-version" --filter "status=running" --quiet | grep -q .; then + echo "PR version container failed to start" + docker logs pr-version + exit 1 + fi + + if ! docker ps --filter "name=main-version" --filter "status=running" --quiet | grep -q .; then + echo "Main version container failed to start" + docker logs main-version + exit 1 + fi + + # Wait for services to be ready + echo "Waiting for services to be ready..." + timeout 60 bash -c 'until curl -f http://localhost:8080/actuator/health 2>/dev/null; do sleep 2; done' || echo "PR version health check failed" + timeout 60 bash -c 'until curl -f http://localhost:8081/actuator/health 2>/dev/null; do sleep 2; done' || echo "Main version health check failed" + - name: Setup Node.js uses: actions/setup-node@v4 with: From 05b6a25507b93d08db82a222b666a9dfcca90a25 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Wed, 23 Jul 2025 08:45:27 +0200 Subject: [PATCH 14/16] remove duplicates and comment out cleaning --- .github/workflows/build-preview.yml | 116 ---------------------------- .github/workflows/pr-preview.yml | 90 ++++++++++----------- 2 files changed, 45 insertions(+), 161 deletions(-) delete mode 100644 .github/workflows/build-preview.yml diff --git a/.github/workflows/build-preview.yml b/.github/workflows/build-preview.yml deleted file mode 100644 index 17f76f8e6..000000000 --- a/.github/workflows/build-preview.yml +++ /dev/null @@ -1,116 +0,0 @@ -name: Build and Deploy Preview - -on: - pull_request: - types: [opened, synchronize] - paths: - - 'src/main/resources/templates/**' - - 'src/main/resources/static/**' - - 'src/main/resources/explanations/**' - -jobs: - build-preview: - runs-on: ubuntu-latest - # Temporarily disabled to avoid duplicate comments with pr-preview.yml - # Both workflows provide similar functionality, pr-preview.yml is more comprehensive - if: false - steps: - - name: Checkout code - uses: actions/checkout@v4 - - - name: Set up JDK 23 - uses: actions/setup-java@v4 - with: - java-version: "23" - distribution: "oracle" - cache: "maven" - - - name: Extract version from pom.xml - id: extract-version - run: | - echo "Extracting version from pom.xml..." - chmod +x ./mvnw - VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout) - DOCKER_VERSION=${VERSION%-SNAPSHOT} - echo "version=$VERSION" >> $GITHUB_OUTPUT - echo "docker_version=$DOCKER_VERSION" >> $GITHUB_OUTPUT - echo "Detected version: $VERSION" - echo "Docker version: $DOCKER_VERSION" - - - name: Build application - run: | - echo "Building WrongSecrets application..." - ./mvnw --no-transfer-progress clean package -DskipTests - echo "Build completed. Checking target directory..." - ls -la target/ - echo "JAR file details:" - ls -la target/*.jar || echo "No JAR files found in target/" - - - name: Build Docker image - run: | - echo "Building Docker image with version ${{ steps.extract-version.outputs.docker_version }}..." - docker build --build-arg argBasedVersion="${{ steps.extract-version.outputs.docker_version }}" -t wrongsecrets-preview . - echo "Docker image built successfully" - docker save wrongsecrets-preview > wrongsecrets-preview.tar - - - name: Upload preview artifact - uses: actions/upload-artifact@v4 - with: - name: wrongsecrets-preview-${{ github.event.number }} - path: wrongsecrets-preview.tar - - - name: Comment on PR with instructions - uses: actions/github-script@v7 - with: - script: | - const prNumber = context.issue.number; - const runId = context.runId; - - const comment = `๐Ÿ”จ **Preview Build Complete!** - - Your changes have been built successfully. - - **๐Ÿ“ฆ Download & Test Locally:** - 1. [๐Ÿ“ Download Docker Image Artifact](https://github.com/${{ github.repository }}/actions/runs/${runId}) (look for \`wrongsecrets-preview-${prNumber}\`) - 2. Load and run the image: - \`\`\`bash - # Download the artifact, extract it, then: - docker load < wrongsecrets-preview.tar - docker run -p 8080:8080 wrongsecrets-preview - \`\`\` - - Then visit: http://localhost:8080 - - **๐Ÿ“ Changed Files in this PR:**`; - - // Get the list of changed files - const { data: files } = await github.rest.pulls.listFiles({ - owner: context.repo.owner, - repo: context.repo.repo, - pull_number: prNumber - }); - - const templateFiles = files.filter(file => - file.filename.includes('templates/') || - file.filename.includes('static/') || - file.filename.includes('explanations/') - ); - - let filesList = ''; - if (templateFiles.length > 0) { - filesList = templateFiles.map(file => `- \`${file.filename}\``).join('\n '); - } else { - filesList = '- No template/static files changed'; - } - - const fullComment = comment + '\n ' + filesList + ` - - --- - Build completed by GitHub Actions`; - - github.rest.issues.createComment({ - issue_number: prNumber, - owner: context.repo.owner, - repo: context.repo.repo, - body: fullComment - }); diff --git a/.github/workflows/pr-preview.yml b/.github/workflows/pr-preview.yml index 82667dfe3..1c7f2b128 100644 --- a/.github/workflows/pr-preview.yml +++ b/.github/workflows/pr-preview.yml @@ -433,34 +433,34 @@ jobs: repo: context.repo.repo, body: comment }); - - cleanup-preview: - runs-on: ubuntu-latest - if: github.event.action == 'closed' - steps: - - name: Log in to GitHub Container Registry - uses: docker/login-action@v3 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - - name: Delete PR container images - run: | - # Delete container images for this PR - PR_NUMBER=${{ github.event.number }} - REPO_LOWER=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]') - - # Get all tags for this PR and delete them - for tag in $(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ - "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions" | \ - jq -r --arg pr "$PR_NUMBER" '.[] | select(.metadata.container.tags[]? | contains($pr)) | .id'); do - - echo "Deleting container version: $tag" - curl -X DELETE \ - -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ - "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions/$tag" - done +#only need to enable if you want to clean up resources on PR close: +# cleanup-preview: +# runs-on: ubuntu-latest +# if: github.event.action == 'closed' +# steps: +# - name: Log in to GitHub Container Registry +# uses: docker/login-action@v3 +# with: +# registry: ghcr.io +# username: ${{ github.actor }} +# password: ${{ secrets.GITHUB_TOKEN }} + +# - name: Delete PR container images +# run: | +# # Delete container images for this PR +# PR_NUMBER=${{ github.event.number }} +# REPO_LOWER=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]') + +# # Get all tags for this PR and delete them +# for tag in $(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ +# "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions" | \ +# jq -r --arg pr "$PR_NUMBER" '.[] | select(.metadata.container.tags[]? | contains($pr)) | .id'); do + +# echo "Deleting container version: $tag" +# curl -X DELETE \ +# -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \ +# "https://api.github.com/orgs/${{ github.repository_owner }}/packages/container/${REPO_LOWER}%2Fwrongsecrets-pr/versions/$tag" +# done # Comment out Render cleanup for now # - name: Cleanup Render service @@ -477,24 +477,24 @@ jobs: # -H "Authorization: Bearer $RENDER_API_KEY" # fi - - name: Comment PR closure - uses: actions/github-script@v7 - with: - script: | - const comment = `๐Ÿงน **Preview Cleanup Complete** + # - name: Comment PR closure + # uses: actions/github-script@v7 + # with: + # script: | + # const comment = `๐Ÿงน **Preview Cleanup Complete** - PR preview resources have been cleaned up: - - โœ… Container images deleted from GitHub Container Registry - - โœ… Artifacts will expire automatically in 30 days + # PR preview resources have been cleaned up: + # - โœ… Container images deleted from GitHub Container Registry + # - โœ… Artifacts will expire automatically in 30 days - Thanks for contributing to WrongSecrets! ๐ŸŽ‰ + # Thanks for contributing to WrongSecrets! ๐ŸŽ‰ - --- - Cleanup completed by GitHub Actions`; + # --- + # Cleanup completed by GitHub Actions`; - github.rest.issues.createComment({ - issue_number: context.issue.number, - owner: context.repo.owner, - repo: context.repo.repo, - body: comment - }); + # github.rest.issues.createComment({ + # issue_number: context.issue.number, + # owner: context.repo.owner, + # repo: context.repo.repo, + # body: comment + # }); From 8fc9b14a46d6da12c1a60b8f0df883847b9f2a96 Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Thu, 24 Jul 2025 06:45:00 +0200 Subject: [PATCH 15/16] add missing rennovate items --- renovate.json | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/renovate.json b/renovate.json index db769d766..9e562c7a8 100644 --- a/renovate.json +++ b/renovate.json @@ -73,6 +73,39 @@ "LBC_VERSION=\"(?.*?)\"" ], "datasourceTemplate": "github-releases" + }, + { + "customType": "regex", + "description": "Track Docker base images in Dockerfiles", + "fileMatch": ["(^|/)Dockerfile[^/]*$"], + "matchStrings": [ + "FROM\\s+(?bellsoft/liberica-openjre-debian):(?[^\\s]+)", + "FROM\\s+(?eclipse-temurin):(?[^\\s]+)" + ], + "datasourceTemplate": "docker" + }, + { + "customType": "regex", + "description": "Track runtime versions in GitHub Actions", + "fileMatch": ["^\\.github/workflows/[^/]+\\.ya?ml$"], + "matchStrings": [ + "node-version:\\s*['\"]?(?[^'\"\\s]+)['\"]?", + "python-version:\\s*['\"]?(?[^'\"\\s]+)['\"]?", + "java-version:\\s*['\"]?(?[^'\"\\s]+)['\"]?" + ], + "depNameTemplate": "node", + "datasourceTemplate": "node-version" + }, + { + "customType": "regex", + "description": "Track jeroenwillemsen/wrongsecrets images in K8s manifests", + "fileMatch": ["(^|/)k8s/.+\\.ya?ml$", "(^|/)[^/]*\\.ya?ml$"], + "matchStrings": [ + "image:\\s*['\"]?jeroenwillemsen/wrongsecrets:(?[^'\"\\s]+)['\"]?", + "image:\\s*['\"]?jeroenwillemsen/wrongsecrets-challenge53:(?[^'\"\\s]+)['\"]?" + ], + "depNameTemplate": "jeroenwillemsen/wrongsecrets", + "datasourceTemplate": "docker" } ], "packageRules": [ @@ -131,6 +164,44 @@ "/google/", "/google-beta/" ] + }, + { + "groupName": "GitHub Actions", + "matchDatasources": ["github-tags"], + "matchPackageNames": [ + "/^actions\//" + ] + }, + { + "groupName": "Docker Actions", + "matchDatasources": ["github-tags"], + "matchPackageNames": [ + "/^docker\//" + ] + }, + { + "groupName": "Java Runtime Images", + "matchDatasources": ["docker"], + "matchPackageNames": [ + "bellsoft/liberica-openjre-debian", + "eclipse-temurin" + ] + }, + { + "groupName": "Runtime Versions", + "schedule": ["before 6am on the first day of the month"], + "matchPackageNames": [ + "node", + "python", + "java" + ] + }, + { + "groupName": "Frontend Dependencies", + "matchPackageNames": [ + "/^playwright/", + "/^@babel\//" + ] } ] } From 50abe2ef6d4309b4b4217e33f1450a580debe2ea Mon Sep 17 00:00:00 2001 From: Jeroen Willemsen Date: Thu, 24 Jul 2025 19:40:02 +0200 Subject: [PATCH 16/16] Update README.md Co-authored-by: Ben de Haan <53901866+bendehaan@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 300ef39be..212154030 100644 --- a/README.md +++ b/README.md @@ -122,7 +122,7 @@ Not sure which setup is right for you? Here's a quick guide: | **I want to...** | **Recommended Setup** | **Challenges Available** | |------------------|----------------------|--------------------------| -| Try it quickly online | [Heroku Demo](https://wrongsecrets.herokuapp.com/) | Basic challenges (1-4, 8, 12-32, 34-43, 49-52, 54-55) | +| Try it quickly online | [Container running on Heroku](https://www.wrongsecrets.com/) | Basic challenges (1-4, 8, 12-32, 34-43, 49-52, 54-55) | | Run locally with Docker | [Basic Docker](#basic-docker-exercises) | Same as above, but on your machine | | Learn Kubernetes secrets | [K8s/Minikube Setup](#basic-k8s-exercise) | Kubernetes challenges (1-6, 8, 12-43, 48-55) | | Practice with cloud secrets | [Cloud Challenges](#cloud-challenges) | All challenges (1-55) |