Quality Gate Check

README.md

@@ -67,18 +67,16 @@ This repository contains the following Cucumber Scenario tests:
 - [BrowserStack](#browserstack)
 
 ---
-## Configuring the maximum parallel test threads for this repository
+## Configuring the maximum parallel test threads for this repository[SDK]
 
-For all the parallel run configuration profiles, tests will run with max parallels in your BrowserStack account, or you can configure the parallel test threads from the terminal
+For all the parallel run configuration profiles, tests will run with max parallels in your BrowserStack account, or you can configure the parallel test threads from the browserstack.yml file
 
-    "-DthreadCount=xx"
+    parallelsPerPlatform
 
 |BrowserStack| onPrem |
 |--|--|
 | browserstack/conf/Run_Single_Test/single.testng.xml|testng.xml
-browserstack/conf/Run_Parallel_Test/parallel.testng.xml |  |
-|browserstack/conf/Run_Local_Test/local.testng.xml|
-|browserstack/conf/Run_Parallel_Test/parallel.testng.xml
+browserstack/conf/Run_Parallel_Test/parallel.testng.xml |
 
 
 
@@ -142,11 +140,8 @@ This infrastructure points to running the tests on your own machine using any br
   set BROWSERSTACK_ACCESS_KEY=<browserstack-access-key>
   ```
 
-  Alternatively, you can also hardcode username and access_key objects in the respective json files:
-  - [single.conf.json](src/test/resources/browserstack/conf/Run_Single_Test/single.conf.json) file
-  - [parallel.conf.json](src/test/resources/browserstack/conf/Run_Parallel_Test/parallel.conf.json) file
-  - [local.conf.json](src/test/resources/browserstack/conf/Run_Local_Test/local.conf.json) file
-  - [local.parallel.json](src/test/resources/browserstack/conf/Run_Local_Parallel/local.parallel.json) file
+  Alternatively, you can also hardcode username and access_key objects in the respective file:
+  - [browserstack.yml](browserstack.yml) file
 
 
 Note:
@@ -158,7 +153,7 @@ Note:
 
 ### Run a specific test/entire test suite in parallel on a single BrowserStack browser
 
-In this section, we will run the tests in parallel on a single browser on Browserstack. Refer to [single.conf.json](src/test/resources/browserstack/conf/Run_Single_Test/single.conf.json) file to change test capabilities for this configuration.
+In this section, we will run the tests in parallel on a single browser on Browserstack. Refer to [browserstack.yml](browserstack.yml) file to change test capabilities for this configuration.
 
 - How to run the test?
 
@@ -178,21 +173,21 @@ In this section, we will run the tests in parallel on a single browser on Browse
   - Note: By default, this execution would run maximum test threads based on the parallel quota on your BrowserStack Account. Thread count can be configured as below based on your requirements.
 
    ```sh
-  mvn test -P scenario-bs "-Dcucumber.filter.tags=@regresssion" "-DthreadCount=3"
+  mvn test -P scenario-bs "-Dcucumber.filter.tags=@regresssion"
   ```
 
 
 ### Run the entire test suite in parallel on multiple BrowserStack browsers
 
-In this section, we will run the tests in parallel on multiple browsers on Browserstack. Refer to the [parallel.conf.json](src/test/resources/browserstack/conf/Run_Parallel_Test/parallel.conf.json) file to change test capabilities for this configuration.
+In this section, we will run the tests in parallel on multiple browsers on Browserstack. Refer to the [browserstack.yml](browserstack.yml) file to change test capabilities for this configuration.
 
 - How to run the test?
 
   To run the entire test suite in parallel on multiple BrowserStack browsers/devices, use the following command:
 
   Maven:
   ```sh
-  mvn test -P suite-cross-bs
+  mvn test -P scenario-bs
   ```
 
 	You can mention any scenario from the feature files using the `-Dcucumber.filter.tags`, tags defined at Feature level Eg. `@users` will run all the scenarios in the [Users Feature](src/test/resources/Features/Users.feature) file in parallel. Likewise `@regression` will run all the scenarios from all the Feature files in parallel across multiple browsers/devices.
@@ -206,7 +201,7 @@ In this section, we will run the tests in parallel on multiple browsers on Brows
   git clone https://github.com/browserstack/browserstack-demo-app
   ``` 
 - Please follow the README.md on the BrowserStack demo application repository to install and start the dev server on localhost.
-- In this section, we will run a single test case to test the BrowserStack Demo app hosted on your local machine i.e. localhost. Refer to the [local.conf.json](src/test/resources/browserstack/conf/Run_Local_Test/local.conf.json) file to change test capabilities for this configuration.
+- In this section, we will run a single test case to test the BrowserStack Demo app hosted on your local machine i.e. localhost. Refer to the [browserstack.yml](browserstack.yml) file to change test capabilities for this configuration.
 - Note: You may need to provide additional BrowserStackLocal arguments to successfully connect your localhost environment with BrowserStack infrastructure. (e.g if you are behind firewalls, proxy or VPN).
 - Further details for successfully creating a BrowserStackLocal connection can be found here:
 
@@ -234,15 +229,15 @@ You can mention any scenario from the feature files using the `-Dcucumber.filter
 
 ### [Web application hosted on internal environment] Run the entire test suite in parallel on multiple BrowserStack browser using BrowserStackLocal
 
-In this section, we will run the test cases to test the internally hosted website in parallel on multiple browsers/devices on Browserstack. Refer to the [local.parallel.json](src/test/resources/browserstack/conf/Run_Local_Parallel/local.parallel.json) file to change test capabilities for this configuration.
+In this section, we will run the test cases to test the internally hosted website in parallel on multiple browsers/devices on Browserstack. Refer to the [browserstack.yml](browserstack.yml) file to change test capabilities for this configuration.
 
 - How to run the test?
 
   To run the entire test suite in parallel on multiple devices and browsers using BrowserStackLocal, use the following command:
 
   Maven:
   ```sh
-  mvn test -P local-cross-bs
+  mvn test -P local-bs
   ```
 
 - Output
@@ -251,7 +246,7 @@ In this section, we will run the test cases to test the internally hosted websit
 
 - Note: By default, this execution would run maximum test threads based on the parallel quota on your BrowserStack Account. Thread count can be configured as below based on your requirements.
   ```sh
-  mvn test -P local-cross-bs "-Dcucumber.filter.tags=@e2e" "-DthreadCount=3"
+  mvn test -P local-bs "-Dcucumber.filter.tags=@e2e"
   ```
 
 

browserstack.yml

@@ -0,0 +1,107 @@
+# =============================
+# Set BrowserStack Credentials
+# =============================
+# Add your BrowserStack userName and accessKey here or set BROWSERSTACK_USERNAME and
+# BROWSERSTACK_ACCESS_KEY as env variables
+userName: YOUR_USERNAME
+accessKey: YOUR_ACCESS_KEY
+
+# ======================
+# BrowserStack Reporting
+# ======================
+# The following capabilities are used to set up reporting on BrowserStack:
+# Set 'projectName' to the name of your project. Example, Marketing Website
+projectName: BrowserStack Cucumber TestNG # Automate - Map SDK
+# Set `buildName` as the name of the job / testsuite being run
+buildName: browserstack build
+# `buildIdentifier` is a unique id to differentiate every execution that gets appended to
+# buildName. Choose your buildIdentifier format from the available expressions:
+# ${BUILD_NUMBER} (Default): Generates an incremental counter with every execution
+# ${DATE_TIME}: Generates a Timestamp with every execution. Eg. 05-Nov-19:30
+# Read more about buildIdentifiers here -> https://www.browserstack.com/docs/automate/selenium/organize-tests
+buildIdentifier: '#${BUILD_NUMBER}' # Supports strings along with either/both ${expression}
+buildTag: "regresson"
+selfHeal: true
+# =======================================
+# Platforms (Browsers / Devices to test)
+# =======================================
+# Platforms object contains all the browser / device combinations you want to test on.
+# Entire list available here -> (https://www.browserstack.com/list-of-browsers-and-platforms/automate)
+platforms:
+  - os: OS X
+    osVersion: Big Sur
+    browserName: Chrome
+    browserVersion: 131.0
+  - os: Windows
+    osVersion: 10
+    browserName: Edge
+    browserVersion: 131.0
+  - deviceName: Samsung Galaxy S2*
+    browserName: chrome # Try 'samsung' for Samsung browser
+    osVersion: [12131415]
+  - deviceName: iPhone 1* 
+    browserName: safari
+    osVersion: [15161718]
+# =======================
+# Parallels per Platform
+# =======================
+# The number of parallel threads to be used for each platform set.
+# BrowserStack's SDK runner will select the best strategy based on the configured value
+#
+# Example 1 - If you have configured 3 platforms and set `parallelsPerPlatform` as 2, a total of 6 (2 * 3) parallel threads will be used on BrowserStack
+#
+# Example 2 - If you have configured 1 platform and set `parallelsPerPlatform` as 5, a total of 5 (1 * 5) parallel threads will be used on BrowserStack
+parallelsPerPlatform: 2
+framework: cucumber-testng #junit,testng,java,cucumber-testng,serenity,cucumber-junit
+source: cucumber-testng:intellij:v1.1.2
+staticWebDriver: true
+browserstackAutomation: true
+# ==========================================
+# BrowserStack Local
+# (For localhost, staging/private websites)
+# ==========================================
+# Set browserStackLocal to true if your website under test is not accessible publicly over the internet
+# Learn more about how BrowserStack Local works here -> https://www.browserstack.com/docs/automate/selenium/local-testing-introduction
+browserstackLocal: false # <boolean> (Default false)
+
+# Options to be passed to BrowserStack local in-case of advanced configurations
+#browserStackLocalOptions:
+  # localIdentifier: # <string> (Default: null) Needed if you need to run multiple instances of local.
+  # forceLocal: false  # <boolean> (Default: false) Set to true if you need to resolve all your traffic via BrowserStack Local tunnel.
+  # Entire list of arguments available here -> https://www.browserstack.com/docs/automate/selenium/manage-incoming-connections
+
+# ===================
+# Debugging features
+# ===================
+debug: true # <boolean> # Set to true if you need screenshots for every selenium command ran
+networkLogs: true # <boolean> Set to true to enable HAR logs capturing
+consoleLogs: errors # <string> Remote browser's console debug levels to be printed (Default: errors)
+# Available options are `disable`, `errors`, `warnings`, `info`, `verbose` (Default: errors)
+
+# Test Observability is an intelligent test reporting & debugging product. It collects data using the SDK. Read more about what data is collected at https://www.browserstack.com/docs/test-observability/references/terms-and-conditions
+# Visit observability.browserstack.com to see your test reports and insights. To disable test observability, specify `testObservability: false` in the key below.
+testObservability: true
+accessibility: true
+accessibilityOptions:
+  wcagVersion: wcag21aaa # Default: wcag21aa
+  includeIssueType:
+    bestPractice: true # Default: false
+    needsReview: true # Default: true
+    experimental: true # Default: true
+  #includeTagsInTestingScope: [ '@users' ] # Tags for test cases to include
+  #excludeTagsInTestingScope: [ 'old', 'deprecated' ] # Tags for test cases to exclude
+
+performance: assert
+percy: true
+percyCaptureMode: auto
+#logLevel: debug
+browserProfiling: true
+
+testOrchestrationOptions:
+  retryTestsOnFailure:
+    enabled: true
+    maxRetries: 2
+  runSmartSelection:
+    enabled: true
+    mode: relevantFirst
+  runPreviouslyFailedFirst: true

check-quality-gate.sh

@@ -0,0 +1,99 @@
+#!/bin/bash
+set -e # Exit immediately if a command exits with a non-zero status.
+
+# --- Configuration ---
+# These values are read from Azure DevOps Pipeline variables.
+project_name="BrowserStack Cucumber TestNG"
+build_name="$BROWSERSTACK_BUILD_NAME"
+username="$BROWSERSTACK_USERNAME"
+access_key="$BROWSERSTACK_ACCESS_KEY"
+
+# --- Script Parameters ---
+max_attempts=20 # Max number of times to poll the API (20 attempts * 30s = 10 minutes)
+build_tags=""   # Optional: Set build tags if needed
+
+# --- Script Functions ---
+
+# Function to URL-encode a string
+sanitize_name() {
+    local name="$1"
+    echo "$name" | sed 's/ /%20/g'
+}
+
+# Function to get the UUID of the latest build
+get_latest_build_uuid() {
+    local sanitized_project_name
+    local sanitized_build_name
+    sanitized_project_name=$(sanitize_name "$project_name")
+    sanitized_build_name=$(sanitize_name "$build_name")
+
+    # API call relies on project and build name, which is most reliable in CI.
+    local url="https://api-automation.browserstack.com/ext/v1/builds/latest?project_name=$sanitized_project_name&build_name=$sanitized_build_name"
+
+    # Add optional build tags to the query if they are set
+    if [ -n "$build_tags" ]; then
+        local sanitized_build_tags
+        sanitized_build_tags=$(sanitize_name "$build_tags")
+        url="$url&build_tags=$sanitized_build_tags"
+    fi
+
+    echo "Querying for build..." >&2
+
+    # Fetch the JSON response. This goes to standard output.
+    curl -s --retry 3 --connect-timeout 10 -u "$username:$access_key" "$url"
+}
+
+# Function to get the status of the Quality Gate for a given build UUID
+get_quality_gate_result() {
+    local build_uuid="$1"
+    local qg_url="https://api-automation.browserstack.com/ext/v1/quality-gates/$build_uuid"
+
+    # Fetch the JSON response. This goes to standard output.
+    curl -s -H --retry 3 --connect-timeout 10 -u "$username:$access_key" "$qg_url"
+}
+
+# --- Main Script Execution ---
+
+echo "--- BrowserStack Quality Gate Check ---" >&2
+
+build_info_json=$(get_latest_build_uuid)
+build_uuid=$(echo "$build_info_json" | jq -r '.build_id')
+
+if [ "$build_uuid" == "null" ] || [ -z "$build_uuid" ]; then
+    echo "Error: Failed to retrieve a valid Build UUID from the API." >&2
+    echo "API Response: $build_info_json" >&2
+    exit 1
+fi
+
+echo "Successfully found Build UUID: $build_uuid" >&2
+echo "Waiting 20 seconds before polling Quality Gate..." >&2
+sleep 20
+
+# Polling loop
+attempt=0
+while [[ $attempt -lt $max_attempts ]]; do
+    attempt=$((attempt + 1))
+    echo "Polling attempt #$attempt..." >&2
+
+    quality_gate_json=$(get_quality_gate_result "$build_uuid")
+    status=$(echo "$quality_gate_json" | jq -r '.status')
+
+    echo "Current status is: '$status'" >&2
+
+    if [ "$status" != "running" ]; then
+        echo "Final Quality Gate Result: $quality_gate_json"
+
+        if [ "$status" == "passed" ]; then
+            echo "✅ Quality Gate Passed" >&2
+            exit 0
+        else
+            echo "❌ Quality Gate Failed" >&2
+            exit 1
+        fi
+    fi
+
+    sleep 30
+done
+
+echo "Error: Timed out waiting for Quality Gate results after $max_attempts attempts." >&2
+exit 1