Skip to content

feat: Add comprehensive k6 load test suite for ToolJet with 15 test scenarios and VU scaling#1

Open
adishM98 wants to merge 2 commits intomainfrom
feat/loading-testing
Open

feat: Add comprehensive k6 load test suite for ToolJet with 15 test scenarios and VU scaling#1
adishM98 wants to merge 2 commits intomainfrom
feat/loading-testing

Conversation

@adishM98
Copy link

Summary

This PR introduces a complete k6 load testing suite for ToolJet, providing comprehensive performance testing coverage across all major features. The suite includes 15 test
scenarios covering authentication, app building, data sources, workspace management, end users, and enterprise features, with support for adjustable virtual user (VU) scaling.

🎯 Motivation

  • Enable performance testing and validation for ToolJet at scale (500-1000 concurrent users)
  • Provide baseline performance metrics for CI/CD pipeline integration
  • Identify bottlenecks and performance issues before production deployment
  • Ensure 0%-0.1% failure rate under realistic load conditions
  • Support capacity planning and infrastructure scaling decisions

📊 What's Included

Test Coverage (15 Scenarios)

Authentication Tests (3)

  • Login/logout journey (500 VUs, 8-10 RPS)
  • User invite & onboarding (500 VUs, 5-8 RPS)
  • Password reset journey (500 VUs, 5-8 RPS)

App Builder Tests (3)

  • App lifecycle journey (500 VUs, 8-10 RPS)
  • Component workflow journey (500 VUs, 6-8 RPS)
  • Multi-page app journey (500 VUs, 5-8 RPS)

Data Source Tests (4)

  • PostgreSQL full journey (500 VUs, 8-10 RPS)
  • MySQL full journey (500 VUs, 8-10 RPS)
  • REST API full journey (500 VUs, 10-12 RPS)
  • All data sources mixed (1000 VUs, 15-18 RPS)

Workspace Tests (2)

  • Workspace setup journey (500 VUs, 5-8 RPS)
  • User management journey (500 VUs, 5-8 RPS)

End User Tests (2)

  • Public app viewing (1000 VUs, 15-20 RPS)
  • Private app viewing (500 VUs, 8-10 RPS)

Enterprise Tests (1)

  • Multi-environment promotion (500 VUs, 5-8 RPS)

Infrastructure

  • Common utilities for authentication, configuration, and helpers
  • Runner scripts (7) with VU adjustment capability via multiplier parameter
  • Comprehensive documentation including API reference, Cypress mapping, and troubleshooting guide
  • Environment configuration with sensible defaults and examples

🔑 Key Features

  1. Complete User Journeys - Tests simulate real user workflows, not isolated API calls
  2. Conservative Load Settings - 2.5x sleep multiplier ensures 0%-0.1% failure rate
  3. Correct Authentication - Uses proper Tj-Workspace-Id header format (critical fix)
  4. VU Scaling - All runner scripts support load adjustment:
    ./runners/run_all_tests.sh 0.5 # 50% VUs for testing
    ./runners/run_all_tests.sh 2.0 # 200% VUs for stress testing
  5. Based on Cypress Tests - Mapped from 93+ existing Cypress E2E test scenarios
  6. Production-Ready - Includes error handling, automatic cleanup, and proper thresholds

📁 File Structure

  ToolJet-k6-tests/
  ├── common/                    # Shared utilities (3 files)
  │   ├── config.js             # Centralized configuration
  │   ├── auth.js               # Authentication helpers
  │   └── helpers.js            # Reusable functions
  ├── docs/                      # Documentation (4 files)
  │   ├── RUNNING_TESTS.md      # How to run tests
  │   ├── API_REFERENCE.md      # Complete API documentation
  │   ├── CYPRESS_MAPPING.md    # Maps k6 tests to Cypress tests
  │   └── TROUBLESHOOTING.md    # Common issues and solutions
  ├── tests/                     # Test scenarios (15 files)
  │   ├── authentication/       # 3 tests
  │   ├── appBuilder/          # 3 tests
  │   ├── dataSources/         # 4 tests
  │   ├── workspace/           # 2 tests
  │   ├── endUser/             # 2 tests
  │   └── enterprise/          # 1 test
  ├── runners/                   # Test suite runners (7 files)
  ├── .env.example              # Environment variable template
  ├── .gitignore                # Git ignore configuration
  └── README.md                 # Main documentation

🚀 Usage

Quick Start

Setup environment

cp .env.example .env

Edit .env with your ToolJet instance details

Run individual test

k6 run tests/authentication/login_logout_journey_500vus.js

Run test suite

./runners/run_all_tests.sh

Run with custom VU scaling

./runners/run_datasource_suite.sh 0.5 # 50% VUs

Configuration

All tests can be configured via .env:
BASE_URL=<site_endpoint>
WORKSPACE_ID=your-workspace-uuid
EMAIL=dev@tooljet.io
PASSWORD=password
SLEEP_MULTIPLIER=2.5 # Adjust for your environment

🎯 Performance Targets

  • Target VUs: 500-1000 depending on test type
  • Expected RPS: 5-20 depending on test complexity
  • Failure Rate: < 0.1%
  • P95 Response Time: < 2000ms

🔧 Technical Details

Critical Fix: Authentication Headers

All tests use the correct authentication header format:
// CORRECT (capital T and W):
headers: { 'Tj-Workspace-Id': workspaceId }

// WRONG (would cause 401 errors):
headers: { 'tj-workspace-id': workspaceId }

Test Pattern

Each test follows a complete user journey:
export default function () {
// 1. Login
let authData = login();

// 2. Perform operations
createApp(authData.authToken, authData.workspaceId, appName);

// 3. Cleanup
deleteApp(authData.authToken, authData.workspaceId, appId);

// 4. Logout
logout(authData.authToken, authData.workspaceId);

// 5. Sleep between iterations
adjustedSleep(120, 60);

}

📚 Documentation

  • RUNNING_TESTS.md - Detailed guide for running tests and interpreting results
  • API_REFERENCE.md - Complete ToolJet API reference with all endpoints used
  • CYPRESS_MAPPING.md - Maps each k6 test to corresponding Cypress tests
  • TROUBLESHOOTING.md - Common issues, solutions, and debugging tips

✅ Testing

All 15 test scenarios have been validated to:

  • Successfully authenticate with ToolJet
  • Execute complete user journeys
  • Clean up test data properly
  • Meet performance targets under load
  • Handle errors gracefully

🔍 Cypress Test Mapping

This suite is based on existing Cypress tests from:

  • cypress/e2e/happyPath/authentication/*.cy.js
  • cypress/e2e/happyPath/appbuilder/*.cy.js
  • cypress/e2e/happyPath/marketplace/data-source/*.cy.js
  • cypress/e2e/happyPath/platform/*.cy.js

See docs/CYPRESS_MAPPING.md for detailed mapping.

🎓 Future Enhancements

Potential additions (out of scope for this PR):

  • Integration with CI/CD pipeline
  • Grafana dashboard for real-time monitoring
  • Performance regression detection
  • Additional test scenarios (workflows, copilot, marketplace plugins)
  • Support for distributed k6 runs

📝 Notes

  • Enterprise tests require ToolJet Enterprise Edition with multi-environment feature
  • Database configuration required for PostgreSQL and MySQL tests (see .env.example)
  • Clean up is automatic but can be verified manually after test runs
  • VU scaling is currently supported via runner script parameters
  • All tests use conservative settings (2.5x sleep multiplier) for 0%-0.1% failure rate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants