Skip to content

Add integration with Diffy for Visual Regression #131

@AlexSkrypnyk

Description

@AlexSkrypnyk

The GHA below is based on the fact the the env is reporting it's deployment AND that there is a VR label added to the PR.

name: Test - Visual regression

on:
  deployment_status:

jobs:
  test-vr:
    runs-on: ubuntu-latest
    
    env:
      TZ: Australia/Melbourne
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Set up PHP
        uses: shivammathur/setup-php@v2

      - name: Retrieve PR number and branch
        id: retrieve_pr_branch
        run: |
          pr="$(echo "${DIFFY_DST_ENV_URL}" | sed -n 's|.*app\.pr-\([0-9]\+\)\..*|\1|p')"
          if [ -n "${pr}" ]; then
            echo "PR number found: ${pr}"
            echo "pr_number=${pr}" >> $GITHUB_OUTPUT            
            echo "is_pr=true" >> $GITHUB_OUTPUT
          else
            echo "No PR number found."
            echo "is_pr=false" >> $GITHUB_OUTPUT
          fi

          branch="$(echo "${DIFFY_DST_ENV_URL}" | sed -n 's|.*app\.\(develop\)\..*|\1|p; s|.*app\.\(master\)\..*|\1|p; s|.*app\.\(release-[^\.]*\)\..*|\1|p')"
          echo "Branch found: ${branch}"
          echo "branch=${branch}" >> $GITHUB_OUTPUT
        env:
          DIFFY_DST_ENV_URL: ${{ github.event.deployment_status.environment_url }}

      - name: Determine if should run Diffy testing
        id: determine_run_diffy
        run: |
          should_run=false
          if [ "${{ steps.retrieve_pr_branch.outputs.is_pr }}" == "true" ]; then
            pr_number=$(echo "${{ steps.retrieve_pr_branch.outputs.pr_number }}" | sed 's/pr-\([0-9]\+\)/\1/')
            labels=$(curl -s -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" "https://api.github.com/repos/${{ github.repository }}/issues/${pr_number}/labels")
            if echo "$labels" | grep -q '"name": "VR"'; then              
              echo "Testing will run for the PR with the VR label."
              should_run=true
            else
              echo "Testing will not run for the PR without the VR label."
              should_run=false
            fi      
          elif [ "${{ steps.retrieve_pr_branch.outputs.branch }}" == "master" ] || [ "${{ steps.retrieve_pr_branch.outputs.branch }}" == "develop" ] || [[ "${{ steps.retrieve_pr_branch.outputs.branch }}" == release-* ]]; then
            echo "Testing will run for the one of the allowed branches."
            should_run=true
          fi
          echo "should_run=${should_run}" >> $GITHUB_OUTPUT

      - name: Run Diffy testing
        if: steps.determine_run_diffy.outputs.should_run == 'true'
        run: |
          temp_env_name="${{ steps.retrieve_pr_branch.outputs.branch }}"
          if [ -z "${temp_env_name}" ]; then
            temp_env_name="pr-${{ steps.retrieve_pr_branch.outputs.pr_number }}"
          fi
          DIFFY_DST_ENV_NAME="${DIFFY_DST_ENV_NAME:-$temp_env_name}"
          
          diffy_name="${DIFFY_SRC_ENV_NAME} $(date +"%d %B") vs ${DIFFY_DST_ENV_NAME} $(date +"%d %B")"
      
          echo "Running Diffy testing "${diffy_name}" for ${DIFFY_SRC_ENV_NAME} and ${DIFFY_DST_ENV_NAME} environments."

          wget https://github.com/diffywebsite/diffy-cli/releases/latest/download/diffy.phar

          php diffy.phar auth:login "${DIFFY_API_KEY}"
          
          php diffy.phar project:compare "${DIFFY_PROJECT_ID}" "${DIFFY_SRC_ENV_NAME}" custom \
            --env2Url="${DIFFY_DST_ENV_URL}" \
            --env2User="${DIFFY_DST_ENV_USER}" \
            --env2Pass="${DIFFY_DST_ENV_PASS}" \
            --name="${diffy_name}"        
        env:
          DIFFY_API_KEY: '${{ secrets.DIFFY_API_KEY }}'
          DIFFY_PROJECT_ID: '${{ vars.DIFFY_PROJECT_ID }}'
          DIFFY_SRC_ENV_NAME: '${{ vars.DIFFY_SRC_ENV_NAME }}'
          DIFFY_DST_ENV_URL: ${{ github.event.deployment_status.environment_url }}
          DIFFY_DST_ENV_USER: '${{ secrets.DIFFY_DST_ENV_USER }}'
          DIFFY_DST_ENV_PASS: '${{ secrets.DIFFY_DST_ENV_PASS }}'
          DIFFY_DST_ENV_NAME: '${{ vars.DIFFY_DST_ENV_NAME }}'

Metadata

Metadata

Assignees

No one assigned

    Labels

    ConfirmedThe issue was triaged and confirmed for development

    Projects

    Status

    Next release

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions