Skip to content

Migrate API Key Mgmt UI tests to Scout #241510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
legrego wants to merge 11 commits into
base: main
Choose a base branch
from
Draft

Migrate API Key Mgmt UI tests to Scout #241510

legrego wants to merge 11 commits into from

Conversation

@legrego
Copy link
Member

@legrego legrego commented Oct 31, 2025 •
edited
Loading

Summary

Migrates API Key Management FTR tests to Scout, with the assistance of an AI Agent following instructions in https://github.com/elastic/kibana/tree/main/src/platform/packages/private/kbn-scout-info/llms.

@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 31, 2025 •
edited
Loading

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!
  • Click to trigger kibana-deploy-cloud-from-pr for this PR!

@legrego legrego requested a review from Copilot October 31, 2025 19:24
Copilot AI reviewed Oct 31, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates API Keys functional tests from the legacy FTR (Functional Test Runner) framework to Scout (Playwright-based testing framework). The migration removes FTR-specific test files and page objects, replacing them with Scout-compatible implementations that leverage Playwright APIs and modern testing patterns.

Key changes:

  • Removed FTR-based tests and page objects for API Keys
  • Created new Scout/Playwright-based tests with equivalent coverage
  • Added Scout API service for API Keys management with comprehensive CRUD operations
  • Enhanced accessibility with announceOnMount props on callout components in the UI

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
x-pack/platform/test/functional/page_objects/index.ts Removed FTR page object provider for API Keys
x-pack/platform/test/functional/page_objects/api_keys_page.ts Deleted FTR-based page object implementation
x-pack/platform/test/functional/apps/api_keys/scout/ui/tests/home_page.spec.ts New Scout test covering API key creation, updating, deletion, and querying
x-pack/platform/test/functional/apps/api_keys/scout/ui/tests/api_keys_security.spec.ts New Scout test for security permissions and access control
x-pack/platform/test/functional/apps/api_keys/scout/ui/playwright.config.ts Scout/Playwright configuration file
x-pack/platform/test/functional/apps/api_keys/scout/ui/fixtures/page_objects/index.ts Scout page object extension mechanism
x-pack/platform/test/functional/apps/api_keys/scout/ui/fixtures/page_objects/api_keys_page.ts Scout-compatible page object with Playwright locators
x-pack/platform/test/functional/apps/api_keys/scout/ui/fixtures/index.ts Scout test fixture definitions
x-pack/platform/test/functional/apps/api_keys/index.ts Deleted FTR test suite index
x-pack/platform/test/functional/apps/api_keys/home_page.ts Deleted FTR test implementation
x-pack/platform/test/functional/apps/api_keys/feature_controls/api_keys_security.ts Deleted FTR security test
x-pack/platform/test/functional/apps/api_keys/config.ts Deleted FTR config file
x-pack/platform/test/functional/apps/api_keys/api_keys_helpers.ts Deleted FTR helper utilities
x-pack/platform/packages/shared/security/api_key_management/src/components/api_key_flyout.tsx Added test subject and accessibility announcements to flyout
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/index.ts Registered API Keys service in Scout API services
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/api_keys/types.ts TypeScript type definitions for API Keys API
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/api_keys/index.ts API Keys service implementation with CRUD and cleanup utilities
src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/api_keys/README.md Comprehensive documentation for API Keys service usage

x-pack/platform/test/functional/apps/api_keys/scout/ui/tests/home_page.spec.ts
await pageObjects.apiKeys.clickSubmitButtonOnApiKeyFlyout();

// Wait for modal to close
await page.waitForTimeout(1000);
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using arbitrary waitForTimeout(1000) is a test anti-pattern. Instead of waiting a fixed 1 second, wait for a specific condition using Playwright's waitFor methods.

Suggested change
await page.waitForTimeout(1000);
await expect.poll(async () => await pageObjects.apiKeys.isApiKeyModalExists()).toBe(false);

Copilot uses AI. Check for mistakes.
x-pack/platform/test/functional/apps/api_keys/scout/ui/tests/home_page.spec.ts Outdated
test('active/expired filter buttons work as expected', async ({ page, pageObjects }) => {
// Click active filter
await pageObjects.apiKeys.clickExpiryFilters('active');
await page.waitForTimeout(500);
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple instances of arbitrary waitForTimeout() calls throughout the test. These should be replaced with proper Playwright wait conditions that check for specific states (e.g., network idle, element visibility) rather than arbitrary delays.

Copilot uses AI. Check for mistakes.
x-pack/platform/test/functional/apps/api_keys/scout/ui/tests/home_page.spec.ts Outdated

// Click elastic option
await page.testSubj.click(`userProfileSelectableOption-elastic_${samlAuth.customRoleName}`);
await page.waitForTimeout(500);
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extensive use of arbitrary waitForTimeout() delays throughout the querying tests. Consider refactoring to use proper wait conditions or adding a helper method that waits for filter application to complete (e.g., waiting for network requests to finish or checking for specific UI state changes).

Copilot uses AI. Check for mistakes.
@elasticmachine
Copy link
Contributor

elasticmachine commented Nov 4, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #33 / saved objects management apis find with kibana index - relationships hasReference and hasReferenceOperator parameters search for a reference

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/scout 189 190 +1
@kbn/scout-oblt 201 202 +1
@kbn/scout-security 205 206 +1
total +3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
searchGettingStarted 155.0KB 155.0KB +32.0B
searchHomepage 146.9KB 147.0KB +32.0B
searchIndices 193.4KB 193.4KB +32.0B
security 486.5KB 486.6KB +119.0B
total +215.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/scout 24 25 +1
Unknown metric groups

API count

id before after diff
@kbn/scout 524 525 +1
@kbn/scout-oblt 538 539 +1
@kbn/scout-security 541 542 +1
total +3

References to deprecated APIs

id before after diff
@kbn/test-suites-xpack-platform 73 76 +3

Unreferenced deprecated APIs

id before after diff
@kbn/scout 17 16 -1

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@legrego @elasticmachine @kibanamachine