Skip to content

feat: SUI-1671 - improve e2e test reliability immediately after deployment#567

Merged
robshakespeare merged 9 commits intomainfrom
SUI-1671_e2e-test-reliability
Apr 16, 2026
Merged

feat: SUI-1671 - improve e2e test reliability immediately after deployment#567
robshakespeare merged 9 commits intomainfrom
SUI-1671_e2e-test-reliability

Conversation

@robshakespeare
Copy link
Copy Markdown
Collaborator

@robshakespeare robshakespeare commented Apr 14, 2026

Updates the e2e tests to wait for the latest version to be deployed and started before the tests commence. Rather than wait for an arbitrary period of time (which could wait for too long unnecessarily), it waits until the API responds with a build timestamp which is on or after a known point in time that confirms the latest version is in use.

Context and background info:

The e2e tests falsely fail when they are automatically ran after deployment. They fail because the e2e tests run immediately after deployment, at which point the deployed API reports that is is up and healthy because the new deployment hasn’t started up yet. Once the function app restarts, the API then temporarily times-out. After a non-determinate period of time, the new version of the API is up and healthy. The changes in this PR ensure that the e2e tests wait for the new version to be deployed and up and running.

…yment

Updates the e2e tests to wait for the new version to be started, before they commence.  Rather than wait for an arbitrary period of time (which could wait for too long unnecessarily), it waits until the API responds with the expected build number.
@sonarqubecloud
Copy link
Copy Markdown

@github-actions
Copy link
Copy Markdown

Summary

Summary
Generated on: 04/15/2026 - 12:01:38
Coverage date: 04/15/2026 - 12:01:27 - 04/15/2026 - 12:01:37
Parser: MultiReport (8x Cobertura)
Assemblies: 5
Classes: 116
Files: 103
Line coverage: 81.7% (4813 of 5886)
Covered lines: 4813
Uncovered lines: 1073
Coverable lines: 5886
Total lines: 9629
Branch coverage: 73.3% (888 of 1210)
Covered branches: 888
Total branches: 1210
Method coverage: Feature is only available for sponsors

Coverage

SUI.Find.Application - 91.1%
Name Line Branch
SUI.Find.Application 91.1% 76.5%
SUI.Find.Application.Constants.Matching.MatchScoreConstants 0%
SUI.Find.Application.Extensions.LoggerActivityExtensions 96.5% 66.6%
SUI.Find.Application.Extensions.OrchestratorRuntimeStatusExtensions 72.7% 37.5%
SUI.Find.Application.Factories.PdsSearch.PdsSearchFactory 100% 100%
SUI.Find.Application.Models.AuditPayloads.PepFetchPayload 100% 100%
SUI.Find.Application.Models.CustodianSearchResultItem 100%
SUI.Find.Application.Models.Fhir.SearchQuery 76.4% 83.3%
SUI.Find.Application.Models.Fhir.SearchResult 100%
SUI.Find.Application.Models.JobResultRecord 100% 50%
SUI.Find.Application.Models.Matching.EncryptedSuidPersonId 100%
SUI.Find.Application.Models.Matching.MatchResult 86.1% 77.7%
SUI.Find.Application.Models.Matching.Metadata 100% 100%
SUI.Find.Application.Models.Matching.PlainPersonId 100%
SUI.Find.Application.Models.SearchResultItem 100% 100%
SUI.Find.Application.Services.FetchRecordService 94.1% 90.5%
SUI.Find.Application.Services.MaskUrlService 100% 100%
SUI.Find.Application.Services.Matching.MatchingEncryptionService 86.6% 71.4%
SUI.Find.Application.Services.Matching.MatchingService 97.8% 88.2%
SUI.Find.Application.Services.MatchPersonOrchestrationService 92.1% 81.2%
SUI.Find.Application.Services.PdsSearch.PdsSearchV1 91.6%
SUI.Find.Application.Services.PdsSearch.SearchQueryBuilder 88.7% 56.2%
SUI.Find.Application.Services.PolicyEnforcementService 92.5% 80.5%
SUI.Find.Application.Services.QueryProvidersService 100% 100%
SUI.Find.Application.Services.SearchResultsService 87% 50%
SUI.Find.Application.Services.SearchService 95.8% 85.1%
SUI.Find.Application.Validation.Matching.PersonDataQualityTranslator 100% 95.1%
SUI.Find.Application.Validation.Matching.PersonSpecificationValidation 98.3% 50%
System.Text.RegularExpressions.Generated 73.3% 50%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F900DA481A13958B
0FDD243B6774DF288C2EB2558EB4EA05262827D57F2E697FC__CommonPhoneSeparatorRege
x_1
75% 66.6%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F900DA481A13958B
0FDD243B6774DF288C2EB2558EB4EA05262827D57F2E697FC__FindWhiteSpaceRegex_0
83.9% 66.6%
SUI.Find.AuditProcessor - 11.2%
Name Line Branch
SUI.Find.AuditProcessor 11.2% 0%
SUI.Find.AuditProcessor.DirectFunctionExecutor 0% 0%
SUI.Find.AuditProcessor.FunctionExecutorAutoStartup 0%
SUI.Find.AuditProcessor.FunctionExecutorHostBuilderExtensions 0%
SUI.Find.AuditProcessor.FunctionMetadataProviderAutoStartup 0%
SUI.Find.AuditProcessor.Functions.QueueAuditAccessTrigger 100%
SUI.Find.AuditProcessor.GeneratedFunctionMetadataProvider 0%
SUI.Find.AuditProcessor.WorkerHostBuilderFunctionMetadataProviderExtension 0%
SUI.Find.Domain - 89.9%
Name Line Branch
SUI.Find.Domain 89.9% 75.8%
SUI.Find.Domain.Models.Result`1 100%
SUI.Find.Domain.ValueObjects.NhsPersonId 96.7% 93.7%
System.Text.RegularExpressions.Generated 87.7% 68.1%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>FAE7CDE050693992
3591B09121AF1D5EBB44979EA3F13DA1D404A81D1B688E40F__Base64UrlRegex_0
87% 70%
SUI.Find.FindApi - 68.8%
Name Line Branch
SUI.Find.FindApi 68.8% 63.7%
SUI.Find.FindApi.DirectFunctionExecutor 0% 0%
SUI.Find.FindApi.FunctionExecutorAutoStartup 0%
SUI.Find.FindApi.FunctionExecutorHostBuilderExtensions 0%
SUI.Find.FindApi.FunctionMetadataProviderAutoStartup 0%
SUI.Find.FindApi.Functions.ActivityFunctions.AuditPepFindActivity 100% 75%
SUI.Find.FindApi.Functions.ActivityFunctions.FilterResultsByPolicyFunction 0%
SUI.Find.FindApi.Functions.ActivityFunctions.GetProvidersFunction 100%
SUI.Find.FindApi.Functions.ActivityFunctions.PersistSearchResultsFunction 100%
SUI.Find.FindApi.Functions.ActivityFunctions.QueryProvidersFunction 88.8% 66.6%
SUI.Find.FindApi.Functions.HttpFunctions.AuthTokenFunction 92.3% 73.9%
SUI.Find.FindApi.Functions.HttpFunctions.CancelSearchFunction 100% 100%
SUI.Find.FindApi.Functions.HttpFunctions.ClaimJobFunction 100% 91.6%
SUI.Find.FindApi.Functions.HttpFunctions.FetchRecordFunction 100% 100%
SUI.Find.FindApi.Functions.HttpFunctions.MatchFunction 88.3% 88.4%
SUI.Find.FindApi.Functions.HttpFunctions.RenewLeaseFunction 97.3% 81.2%
SUI.Find.FindApi.Functions.HttpFunctions.SearchFunction 93.4% 57.1%
SUI.Find.FindApi.Functions.HttpFunctions.SearchFunctionV2 100% 76.9%
SUI.Find.FindApi.Functions.HttpFunctions.SearchResultsFunction 100% 100%
SUI.Find.FindApi.Functions.HttpFunctions.SearchResultsV2Function 100% 90%
SUI.Find.FindApi.Functions.HttpFunctions.SearchStatusFunction 100% 100%
SUI.Find.FindApi.Functions.HttpFunctions.SubmitJobResultsFunction 97.8% 83.3%
SUI.Find.FindApi.Functions.HttpFunctions.WorkAvailableFunction 100% 90%
SUI.Find.FindApi.Functions.OrchestratorFunctions.SearchOrchestrator 100% 100%
SUI.Find.FindApi.Functions.QueueFunctions.JobResultHandlerFunction 100% 75%
SUI.Find.FindApi.Functions.QueueFunctions.QueueSearchJobTrigger 100% 83.3%
SUI.Find.FindApi.Functions.TimerFunctions.ClearDataFunction 81.1% 66.6%
SUI.Find.FindApi.GeneratedFunctionMetadataProvider 0%
SUI.Find.FindApi.Models.SearchJob 100%
SUI.Find.FindApi.Models.SearchJobV2 100%
SUI.Find.FindApi.Models.SearchResults 100%
SUI.Find.FindApi.Models.SearchResultsV2 100%
SUI.Find.FindApi.Utility.BuildTimestampUtility 100% 50%
SUI.Find.FindApi.Utility.HttpResponseUtility 100%
SUI.Find.FindApi.Validators.StartSearchRequestValidator 90.9% 83.3%
SUI.Find.FindApi.WorkerExtensionStartupCodeExecutor 0%
SUI.Find.FindApi.WorkerHostBuilderFunctionMetadataProviderExtension 0%
System.Text.RegularExpressions.Generated 72.1% 50%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>FD7437C968E83D9F
B295AF98F79FFD766C81446A1B277863116E759874FC6C7C2__ExtractBuildTimestampReg
ex_0
71% 50%
SUI.Find.Infrastructure - 92%
Name Line Branch
SUI.Find.Infrastructure 92% 79.8%
SUI.Find.Infrastructure.Clients.AuditQueueClient 100%
SUI.Find.Infrastructure.Clients.JobResultsQueueClient 100% 100%
SUI.Find.Infrastructure.Configuration.JobClaimConfig 100%
SUI.Find.Infrastructure.Extensions.HostApplicationBuilderOpenTelemetryExten
sions
89.7% 87.5%
SUI.Find.Infrastructure.Factories.Fhir.SearchParamsFactory 57.1% 8.3%
SUI.Find.Infrastructure.Handlers.JobResultHandler 94.7% 92.5%
SUI.Find.Infrastructure.Models.CachedToken 100% 50%
SUI.Find.Infrastructure.Repositories.JobRepository.JobKeys 100%
SUI.Find.Infrastructure.Repositories.JobRepository.JobRepository 91.6% 100%
SUI.Find.Infrastructure.Repositories.SearchResultEntryStorage.SearchResultE
ntryKeys
100% 50%
SUI.Find.Infrastructure.Repositories.SearchResultEntryStorage.SearchResultE
ntryRepository
80.2% 100%
SUI.Find.Infrastructure.Repositories.SuiCustodianRegister.RegisterKeys 100%
SUI.Find.Infrastructure.Repositories.SuiCustodianRegister.SuiCustodianRegis
terRepository
90% 100%
SUI.Find.Infrastructure.Repositories.WorkItemJobCountRepository.WorkItemJob
Count
100%
SUI.Find.Infrastructure.Repositories.WorkItemJobCountRepository.WorkItemJob
CountKeys
100%
SUI.Find.Infrastructure.Repositories.WorkItemJobCountRepository.WorkItemJob
CountRepository
100% 100%
SUI.Find.Infrastructure.Services.AuditStorageTableService 100%
SUI.Find.Infrastructure.Services.AzureKeyVaultSecretService 0%
SUI.Find.Infrastructure.Services.BuildCustodianRequestsService 100% 83.3%
SUI.Find.Infrastructure.Services.Fhir.FhirAuthTokenService 89.9% 70.8%
SUI.Find.Infrastructure.Services.Fhir.FhirService 97.7% 75%
SUI.Find.Infrastructure.Services.HashService 100% 100%
SUI.Find.Infrastructure.Services.JobClaimService 89.4% 71.4%
SUI.Find.Infrastructure.Services.JobProcessorService 100% 93.7%
SUI.Find.Infrastructure.Services.JobQueueService 100% 100%
SUI.Find.Infrastructure.Services.JobSearchService 100% 87.5%
SUI.Find.Infrastructure.Services.JobWindowStartService 100%
SUI.Find.Infrastructure.Services.JwtTokenService 100% 50%
SUI.Find.Infrastructure.Services.MockAuthStoreService 68.5% 50%
SUI.Find.Infrastructure.Services.MockCustodianService 98.1% 81.2%
SUI.Find.Infrastructure.Services.MockMatchRepository 100% 90%
SUI.Find.Infrastructure.Services.OutboundAuthService 90.1% 81.2%
SUI.Find.Infrastructure.Services.PersonIdEncryptionService 75.4% 83.3%
SUI.Find.Infrastructure.Services.UrlStorageTableService 100% 100%
SUI.Find.Infrastructure.TableKeyNormaliser 100% 100%
SUI.Find.Infrastructure.Utility.BuildCustodianHttpRequest 100% 90.9%
SUI.Find.Infrastructure.Utility.ProviderHttpClient 74.1% 100%

@robshakespeare robshakespeare marked this pull request as ready for review April 15, 2026 12:24
@robshakespeare robshakespeare enabled auto-merge (squash) April 15, 2026 12:25
Copy link
Copy Markdown
Collaborator

@udara-jay udara-jay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! thanks for doing this! 👍

@robshakespeare robshakespeare merged commit 0a5599a into main Apr 16, 2026
12 checks passed
@robshakespeare robshakespeare deleted the SUI-1671_e2e-test-reliability branch April 16, 2026 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants