Add test coverage for cache scitoken overrides using Xrootd.ScitokensConfig #3028

Copilot · 2026-01-23T23:55:12Z

Adds comprehensive test coverage demonstrating that caches can serve cached objects during origin downtime using the existing Xrootd.ScitokensConfig parameter.

Changes

Test Coverage: Added TestCacheScitokensConfigOverride E2E test that demonstrates:

Sets up a full federation with private reads and pulls a file through the cache
Simulates origin downtime by removing namespace ads from the cache
Triggers cache authz refresh by overwriting Xrootd.ScitokensConfig with an unrelated issuer
Verifies data is no longer accessible through the cache
Triggers another authz refresh with proper authorization for the test prefix
Verifies cached object is now accessible even without origin

Key Finding: The functionality to override scitoken configuration for caches already exists via Xrootd.ScitokensConfig, which is marked as usable by both cache and origin components. No new parameters or implementation changes were needed.

Example Configuration

Administrators can use the existing Xrootd.ScitokensConfig parameter to provide override configurations:

# /path/to/scitokens.cfg
[Global]
audience = https://wlcg.cern.ch/jwt/v1/any

[Issuer SiteIssuer]
issuer = https://site.example.com
base_path = /protected/data
default_user = xrootd

With Xrootd.ScitokensConfig=/path/to/scitokens.cfg, the cache merges these issuers with auto-generated ones, enabling continued service when upstream origins are unreachable. The parameter is already runtime reloadable through the existing maintenance loop.

Original prompt

This section details on the original issue you should resolve

<issue_title>Allow scitoken overrides in the cache</issue_title>
<issue_description>ONLY is the cache is unregistered, we should allow the administrator to specify a scitoken overrides file that gets merged into the calculated scitoken file.

This is analogous to what is done for the origin.

When adding the new configuration parameter, mark it as runtime reloadable. Make sure the file to use (if a file is used) can indeed be changed at runtime.

The goal is to have this be a mechanism where specific caches can continue to serve objects even when the origin is in downtime. Add an integration test along those lines.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes Allow scitoken overrides in the cache #3027

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

bbockelm

It looks like Xrootd.ScitokensConfig works for both caches and origins, meaning this functionality already exists. Let's focus on providing test coverage for this fact in cache mode. I like having both a unit test and an e2e test as is done in this PR.

I'd additionally like a new E2E test that:

Sets up a full federation and pulls a file through the cache.
Places the origin in downtime and removes corresponding namespace ads.
Triggers a cache authz refresh. This can be done by overwriting the Xrootd.ScitokensConfig file with some new issuer covering an unrelated path. You can verify the refresh by seeing new data show up in the corresponding generated file.
Verify you can no longer access the data through the cache.
Trigger another cache authz refresh, this time adding the authorization information explicitly for the test prefix.
Verify you can now access the cached object, even with the origin offline.

This new test will demonstrate that the Xrootd.ScitokensConfig override is the only thing needed to access offline data in a cache.

bbockelm · 2026-01-26T20:58:03Z

e2e_fed_tests/cache_scitoken_override_test.go

+
+/***************************************************************
+*
+* Copyright (C) 2025, Pelican Project, Morgridge Institute for Research


Fixed in commit 0bdf17e

bbockelm · 2026-01-26T21:02:37Z

docs/parameters.yaml

 default: false
 components: ["cache"]
 ---
+name: Cache.ScitokensUserOverride


I don't like this name; ScitokensUserOverride implies this is done by the user (it's done by the administrator) and we tend to suffix files with Location. How about:

Cache.ScitokensConfigLocation

I would note that there's a parameter named Xrootd.ScitokensConfig marked as usable by the cache... is this already done?

You're right - Xrootd.ScitokensConfig already provides this functionality for both cache and origin. Removed the new parameter and reverted to testing the existing functionality in commit 0bdf17e

bbockelm · 2026-01-26T21:03:34Z

docs/parameters.yaml

+  Any issuers defined in this file will be merged with the cache's auto-generated issuers. This is useful for
+  site-local caches that need to continue serving objects during origin downtime.
+
+  Note: This parameter is runtime reloadable. The cache's maintenance loop periodically regenerates the scitokens


This note is unnecessary.

Removed in commit 0bdf17e

bbockelm · 2026-01-26T21:23:10Z

e2e_fed_tests/cache_scitoken_override_test.go

+
+	// Create a scitoken override file for the cache
+	overrideFile := filepath.Join(tmpDir, "cache-override.cfg")
+	require.NoError(t, param.Set(param.Cache_ScitokensUserOverride.GetName(), overrideFile))


It looks like Xrootd.ScitokensConfig already implements this functionality, it's just not tested. Remove the new configuration code and test to see if the old knobs work as expected.

Correct - removed the new configuration code and created a comprehensive E2E test using the existing Xrootd.ScitokensConfig functionality in commit 0bdf17e

bbockelm · 2026-01-26T21:23:40Z

xrootd/authorization.go


+	// If the cache is in site-local mode and a user override file is specified,
+	// load and merge the override issuers into the configuration
+	if param.Cache_EnableSiteLocalMode.GetBool() {


I think this can all be removed -- it looks like the existing Xrootd.ScitokensConfig suffices.

Agreed and removed in commit 0bdf17e

Copilot · 2026-01-26T21:34:44Z

It looks like Xrootd.ScitokensConfig works for both caches and origins, meaning this functionality already exists. Let's focus on providing test coverage for this fact in cache mode. I like having...

Added comprehensive E2E test as requested in commit 0bdf17e. The test:

Sets up federation and pulls file through cache
Removes namespace ads to simulate origin downtime
Triggers cache authz refresh with unrelated issuer
Verifies data is no longer accessible
Triggers another refresh with proper authorization for test prefix
Verifies cached object is accessible despite "offline" origin

The test demonstrates that Xrootd.ScitokensConfig override is sufficient for accessing offline data in a cache.

This helps refresh information quickly in unit tests. Includes a "floor" to prevent this from happening too quickly in production.

If the director failed - but the user specified a cache to use - continue trying the request against that cache. Use the default token; in this case, it will not select the right token automatically. This allows a cache to be leveraged and Pelican to work while everything else in the federation is 100% offline.

Shows that data can be accessed at a cache even when the origin is offline!

Copilot AI assigned Copilot and bbockelm Jan 23, 2026

Copilot started work on behalf of bbockelm January 23, 2026 23:55 View session

Copilot AI changed the title ~~[WIP] Add support for scitoken overrides in the cache~~ Add scitoken override support for site-local caches Jan 24, 2026

Copilot AI requested a review from bbockelm January 24, 2026 00:17

Copilot finished work on behalf of bbockelm January 24, 2026 00:17

bbockelm requested changes Jan 26, 2026

View reviewed changes

Copilot started work on behalf of bbockelm January 26, 2026 21:31 View session

Copilot AI changed the title ~~Add scitoken override support for site-local caches~~ Add test coverage for cache scitoken overrides using Xrootd.ScitokensConfig Jan 26, 2026

Copilot AI requested a review from bbockelm January 26, 2026 21:35

Copilot finished work on behalf of bbockelm January 26, 2026 21:35

bbockelm and others added 3 commits January 26, 2026 21:03

Automatically refresh the namespaces when scitokens.cfg changes

3d27741

This helps refresh information quickly in unit tests. Includes a "floor" to prevent this from happening too quickly in production.

Provide an E2E cache test for offline origins

996bf60

Shows that data can be accessed at a cache even when the origin is offline!

bbockelm force-pushed the copilot/allow-scitoken-overrides-cache branch from 8b9d74c to 996bf60 Compare January 27, 2026 02:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test coverage for cache scitoken overrides using Xrootd.ScitokensConfig #3028

Add test coverage for cache scitoken overrides using Xrootd.ScitokensConfig #3028

Uh oh!

Copilot AI commented Jan 23, 2026 •

edited

Loading

Uh oh!

bbockelm left a comment

Uh oh!

bbockelm Jan 26, 2026

Uh oh!

Copilot AI Jan 26, 2026

Uh oh!

bbockelm Jan 26, 2026

Uh oh!

Copilot AI Jan 26, 2026

Uh oh!

bbockelm Jan 26, 2026

Uh oh!

Copilot AI Jan 26, 2026

Uh oh!

bbockelm Jan 26, 2026

Uh oh!

Copilot AI Jan 26, 2026

Uh oh!

bbockelm Jan 26, 2026

Uh oh!

Copilot AI Jan 26, 2026

Uh oh!

Copilot AI commented Jan 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add test coverage for cache scitoken overrides using Xrootd.ScitokensConfig #3028

Are you sure you want to change the base?

Add test coverage for cache scitoken overrides using Xrootd.ScitokensConfig #3028

Uh oh!

Conversation

Copilot AI commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Example Configuration

Comments on the Issue (you are @copilot in this section)

Uh oh!

bbockelm left a comment

Choose a reason for hiding this comment

Uh oh!

bbockelm Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

bbockelm Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

bbockelm Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

bbockelm Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

bbockelm Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Jan 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 23, 2026 •

edited

Loading