Skip to content

Introduce API Tokens with cluster_permissions and index_permissions directly associated with the token#5443

Open
cwperks wants to merge 38 commits intoopensearch-project:mainfrom
cwperks:feature/api-tokens-cwperx
Open

Introduce API Tokens with cluster_permissions and index_permissions directly associated with the token#5443
cwperks wants to merge 38 commits intoopensearch-project:mainfrom
cwperks:feature/api-tokens-cwperx

Conversation

@cwperks
Copy link
Member

@cwperks cwperks commented Jun 25, 2025

Description

Re-basing #5225 with the latest changes from main.

This PR introduces API Tokens — a new capability in the Security plugin that allows security admins to issue long-lived, scoped tokens and associate permissions directly with the token.

How it works

API Tokens are opaque tokens with the format os_<random>. When a token is created, a SHA-256 hash of the plaintext token is stored in a system index, .opensearch_security_api_tokens. The plaintext token is returned once at creation time and never stored. On each request, the incoming token is hashed and looked up in an in-memory cache populated from the index.

Tokens are authenticated via the Authorization: ApiKey <token> header.

What is novel about this approach compared to OBO tokens is that permissions are scoped directly to the token rather than derived from the issuing user's roles. An admin can issue a token with only the permissions it needs — for example, read-only access to a single index — regardless of the admin's own permissions. This enforces the principle of least privilege and is a key building block toward deprecating Roles Injection, the current practice for how plugins run async jobs with user-scoped permissions.


API Reference

Create API Token

POST /_plugins/_security/api/apitokens

Request:

{
  "name": "my-token",
  "cluster_permissions": ["cluster:monitor/health"],
  "index_permissions": [
    {
      "index_pattern": ["logs-*"],
      "allowed_actions": ["indices:data/read/search"]
    }
  ],
  "expiration": 1800000
}

Response:

{
  "id": "Nd_pMRWeAC93ZGMhRa5CxX",
  "token": "os_abc123..."
}

The id is used to manage the token, such as listing or revoking it. The plaintext token is returned once and never stored — save it immediately.

List API Tokens

GET /_plugins/_security/api/apitokens

Response:

[
  {
    "id": "Nd_pMRWeAC93ZGMhRa5CxX",
    "name": "my-token",
    "iat": 1742000000000,
    "expiration": 1800000,
    "cluster_permissions": ["cluster:monitor/health"],
    "index_permissions": [
      {
        "index_pattern": ["logs-*"],
        "allowed_actions": ["indices:data/read/search"]
      }
    ]
  }
]
Revoke API Token

DELETE /_plugins/_security/api/apitokens/{id}

Response:

{
  "message": "Token Nd_pMRWeAC93ZGMhRa5CxX deleted successfully."
}

Revocation takes effect immediately — any in-flight requests using the token will be rejected after the cache is refreshed across nodes.

Using a Token

Pass the token in the Authorization header using the ApiKey scheme:

Authorization: ApiKey os_abc123...

Example — search a permitted index:

GET /logs-2025/_search
Authorization: ApiKey os_abc123...

Response:

{
  "hits": {
    "total": { "value": 3, "relation": "eq" },
    "hits": [ ... ]
  }
}

Example — attempt a forbidden action:

DELETE /logs-2025
Authorization: ApiKey os_abc123...

Response:

{
  "error": {
    "type": "security_exception",
    "reason": "no permissions for [indices:admin/delete]"
  },
  "status": 403
}

Issues Resolved

Partially resolves #4009, limited to security admins in the initial release.

Check List

  • New functionality includes testing
  • New functionality has been documented
  • New Roles/Permissions have a corresponding security dashboards plugin PR
  • API changes companion pull request created
  • Commits are signed per the DCO using --signoff

derek-ho and others added 23 commits November 14, 2024 10:47
…00 tokens outstanding (opensearch-project#5147)

Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
@cwperks cwperks requested a review from willyborankin as a code owner June 25, 2025 17:56
@cwperks cwperks changed the title Feature/api tokens cwperx Introduce API Tokens with cluster_permissions and index_permissions directly associated with the token Jun 25, 2025
Signed-off-by: Craig Perkins <cwperx@amazon.com>
@codecov
Copy link

codecov bot commented Jun 25, 2025

Codecov Report

❌ Patch coverage is 81.43939% with 98 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.97%. Comparing base (3661e7b) to head (a0876e4).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...arch/security/action/apitokens/ApiTokenAction.java 82.35% 18 Missing and 3 partials ⚠️
...opensearch/security/action/apitokens/ApiToken.java 82.79% 9 Missing and 7 partials ⚠️
...ecurity/action/apitokens/ApiTokenIndexHandler.java 82.14% 10 Missing ⚠️
.../security/action/apitokens/ApiTokenRepository.java 89.79% 7 Missing and 3 partials ⚠️
...pensearch/security/http/ApiTokenAuthenticator.java 81.25% 7 Missing and 2 partials ⚠️
...ction/apitokens/TransportApiTokenUpdateAction.java 65.00% 7 Missing ⚠️
...urity/action/apitokens/ApiTokenUpdateResponse.java 28.57% 5 Missing ⚠️
...ecurity/authtoken/jwt/ExpiringBearerAuthToken.java 0.00% 5 Missing ⚠️
...curity/action/apitokens/ApiTokenUpdateRequest.java 33.33% 4 Missing ⚠️
...search/security/securityconf/impl/v7/ConfigV7.java 78.57% 3 Missing ⚠️
... and 6 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #5443      +/-   ##
==========================================
+ Coverage   73.82%   73.97%   +0.14%     
==========================================
  Files         439      450      +11     
  Lines       27122    27640     +518     
  Branches     4025     4063      +38     
==========================================
+ Hits        20024    20447     +423     
- Misses       5192     5270      +78     
- Partials     1906     1923      +17     
Files with missing lines Coverage Δ
...ecurity/action/apitokens/ApiTokenUpdateAction.java 100.00% <100.00%> (ø)
...ensearch/security/compliance/ComplianceConfig.java 89.80% <100.00%> (+0.70%) ⬆️
...rg/opensearch/security/dlic/rest/api/Endpoint.java 100.00% <100.00%> (ø)
...dlic/rest/api/RestApiAdminPrivilegesEvaluator.java 74.54% <100.00%> (+0.47%) ⬆️
...h/security/privileges/PrivilegesEvaluatorImpl.java 83.09% <100.00%> (+0.24%) ⬆️
...ch/security/securityconf/DynamicConfigFactory.java 66.66% <100.00%> (+0.21%) ⬆️
...ch/security/securityconf/DynamicConfigModelV7.java 69.66% <100.00%> (+1.05%) ⬆️
...g/opensearch/security/ssl/util/ExceptionUtils.java 45.83% <100.00%> (+2.35%) ⬆️
...g/opensearch/security/support/ConfigConstants.java 77.27% <ø> (ø)
...a/org/opensearch/security/util/AuthTokenUtils.java 66.66% <100.00%> (+4.16%) ⬆️
... and 16 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Collaborator

@nibix nibix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Cool to see API tokens going forward.

One general question: I am wondering whether we also need to modify the DLS/FLS code to be API token aware. Otherwise, the deny-by-default policy might block index access.

Comment on lines +234 to +237
pluginIdToActionPrivileges.put(
entry.getKey(),
new SubjectBasedActionPrivileges(entry.getValue(), flattenedActionGroups)
);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this, we might need to make the pluginIdToActionPrivileges HashMap thread-safe, for example, by converting it into a ConcurrentHashMap.

Alternatively, we could have two HashMaps, one for plugins and one for API tokens.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i vote for single thread-safe map


private final Map<String, RoleV7> jtis = new ConcurrentHashMap<>();

void reloadApiTokensFromIndex(ActionListener<Void> listener) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this is getting called when an update action is received. I think I did not see anything regarding initial node startup. Did I miss something?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache seems to be update only when a token is created or deleted. We should add one that loads on node bootstrap.

Comment on lines +199 to +205
if (tokenCount >= 100) {
sendErrorResponse(
channel,
RestStatus.TOO_MANY_REQUESTS,
"Maximum limit of 100 API tokens reached. Please delete existing tokens before creating new ones."
);
return;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be a quite low limit. Is there a reason for that?

Comment on lines +128 to +138
public void createApiToken(
String name,
List<String> clusterPermissions,
List<ApiToken.IndexPermission> indexPermissions,
Long expiration,
ActionListener<String> listener
) {
apiTokenIndexHandler.createApiTokenIndexIfAbsent(ActionListener.wrap(() -> {
ExpiringBearerAuthToken token = securityTokenManager.issueApiToken(name, expiration);
ApiToken apiToken = new ApiToken(name, clusterPermissions, indexPermissions, expiration);
apiTokenIndexHandler.indexTokenMetadata(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the parameter name an identifier for the token? Do we need uniqueness guarantees for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea it is a human readable identifier and uniqueness is not enforced...similar to github personal access tokens.

private final List<TokenListener> tokenListener = new ArrayList<>();
private static final Logger log = LogManager.getLogger(ApiTokenRepository.class);

private final Map<String, RoleV7> jtis = new ConcurrentHashMap<>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about changing RoleV7 into SubjectBasedActionPrivileges here? PrivilegesEvaluator could then retrieve these instances just from here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. It might not be straightforward since JTIs seem to be populated on getTokenMetadata request and there doesn't seem to be a way to pass flattenedActionGroups to that call. I may be seeing the complete picture here, but something like updateJTIs(FlattenedAGs) from PrivilegeEvaluator to update jtis and then use that to populate pluginIdToActionPrivileges which will then be used to create PrivilegeEvalContext?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea I don't think its straightforward, but we should capture this as a refactor that would be good to have

Comment on lines +109 to +111
public Map<String, RoleV7> getJtis() {
return jtis;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, this map should be kept private and managed only by this class.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made package-private and annotated as @VisibleForTesting

public XContentBuilder toXContent(XContentBuilder xContentBuilder, Params params) throws IOException {
xContentBuilder.startObject();
xContentBuilder.field("enabled", enabled);
xContentBuilder.field("signing_key", signing_key);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a hs512 key?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it fails in ApiTokenAuthenticator if it is anything lower than 512 bits.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think at some point in the future we should look to support the opensearch-keystore for values referenced in the security config

Comment on lines +199 to +208
public Boolean isRequestAllowed(final SecurityRequest request) {
Matcher matcher = PATTERN_PATH_PREFIX.matcher(request.path());
final String suffix = matcher.matches() ? matcher.group(2) : null;
if (isAccessToRestrictedEndpoints(request, suffix)) {
final OpenSearchException exception = ExceptionUtils.invalidUsageOfApiTokenException();
log.error(exception.toString());
return false;
}
return true;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a fragile way to deny certain endpoints, especially with the hardcoded path prefixes.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, do I understand it correctly that the goal of this code is that we cannot call the "issue API token" API with API tokens?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't rest-admin-only restriction block access to endpoints anyway?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea the purpose is to prevent using API Tokens to issue API Tokens. Similar to how OBO Tokens cannot be used to issue new OBO Tokens. Any ideas on how to refactor this?

@DarshitChanpura DarshitChanpura mentioned this pull request Jun 26, 2025
5 tasks
Copy link
Member

@DarshitChanpura DarshitChanpura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @cwperks for taking this over. Left some comments around testing and general usage.

Comment on lines +45 to +66
public ApiToken(String name, List<String> clusterPermissions, List<IndexPermission> indexPermissions, Long expiration) {
this.creationTime = Instant.now();
this.name = name;
this.clusterPermissions = clusterPermissions;
this.indexPermissions = indexPermissions;
this.expiration = expiration;
}

public ApiToken(
String name,
List<String> clusterPermissions,
List<IndexPermission> indexPermissions,
Instant creationTime,
Long expiration
) {
this.name = name;
this.clusterPermissions = clusterPermissions;
this.indexPermissions = indexPermissions;
this.creationTime = creationTime;
this.expiration = expiration;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:
can be combined into 1 constructor:

public ApiToken(
            String name,
            List<String> clusterPermissions,
            List<IndexPermission> indexPermissions,
            Instant creationTime,   // nullable
            Long expiration) {

        this.name = name;
        this.clusterPermissions = clusterPermissions;
        this.indexPermissions = indexPermissions;
        this.creationTime = (creationTime != null) ? creationTime : Instant.now();
        this.expiration = expiration;   
    }

Can the name field be null?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the first constructor and made it so that creationTime should be supplied by the caller.


public ApiToken(String name, List<String> clusterPermissions, List<IndexPermission> indexPermissions, Long expiration) {
this.creationTime = Instant.now();
this.name = name;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be UUID instead of name? or does the name field have any association with users?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name is not necessarily unique per token, but it is unique per active token. i.e. on a token's expiry you would be able to regenerate a new one with the same perms.

The metadata which includes the perms for the token is stored as a doc in the .opensearch_security_api_tokens index

private final List<TokenListener> tokenListener = new ArrayList<>();
private static final Logger log = LogManager.getLogger(ApiTokenRepository.class);

private final Map<String, RoleV7> jtis = new ConcurrentHashMap<>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. It might not be straightforward since JTIs seem to be populated on getTokenMetadata request and there doesn't seem to be a way to pass flattenedActionGroups to that call. I may be seeing the complete picture here, but something like updateJTIs(FlattenedAGs) from PrivilegeEvaluator to update jtis and then use that to populate pluginIdToActionPrivileges which will then be used to create PrivilegeEvalContext?

public XContentBuilder toXContent(XContentBuilder xContentBuilder, Params params) throws IOException {
xContentBuilder.startObject();
xContentBuilder.field("enabled", enabled);
xContentBuilder.field("signing_key", signing_key);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it fails in ApiTokenAuthenticator if it is anything lower than 512 bits.

}

@Test
public void issueApiToken_success() throws Exception {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should add a test for failure path

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added negative tests

Settings settings = Settings.builder().put("signing_key", tooShortKey).build();
final Throwable exception = assertThrows(OpenSearchException.class, () -> { new JwtVendor(settings); });

assertThat(exception.getMessage(), containsString("The secret length must be at least 256 bits"));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't signing key at minimum of 512 bits, based on ApiTokenAuthenticator.MINIMUM_SIGNING_KEY_BIT_LENGTH

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message is coming from nimbus and we have a separate and stronger check then nimbus does.

throw new RuntimeException(ex);
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should add test to check for index permissions, i.e. ability to search from, write to an index.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should do this before we merge this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do have testApiTokenWithIndexPermissions_canSearchAllowedIndex

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a test to test out write perms as well

@ztomaszewska
Copy link

Hi @cwperks ! First I wanted to thank you for your work on this PR. Really appreciate effort you put it in to move forward with this feature! Just wanted to ask whether is there any plan to soon merge your changes? Is there any blocking issues/ any help you would need? It would be really cool to have API tokens feature working :)

@cwperks
Copy link
Member Author

cwperks commented Nov 25, 2025

I'll resolve the conflicts ASAP and try to push this forward for V1.

Its important to know that this PR will still have a lot of limitations, but paves the way for expansion. For instance, in this PR only the admin can issue tokens.

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Copy link
Member

@DarshitChanpura DarshitChanpura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for picking this up @cwperks . Left a few comments. Main comment is addition of e2e test which checks authn+authz with API token,


// First check token count
apiTokenRepository.getTokenCount(ActionListener.wrap(tokenCount -> {
if (tokenCount >= 100) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this value configurable with default to 100?
maybe maxed at 1k active tokens?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. making this configurable.

updateRequest,
ActionListener.wrap(
updateResponse -> listener.onResponse(response),
exception -> listener.onFailure(new ApiTokenException("Failed to refresh cache", exception))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this message should be "Failed to update API token"

CHANGELOG.md Outdated
* Create a mechanism for plugins to explicitly declare actions they need to perform with their assigned PluginSubject ([#5341](https://github.com/opensearch-project/security/pull/5341))
* Moves OpenSAML jars to a Shadow Jar configuration to facilitate its use in FIPS enabled environments ([#5400](https://github.com/opensearch-project/security/pull/5404))
* Replaced the standard distribution of BouncyCastle with BC-FIPS ([#5439](https://github.com/opensearch-project/security/pull/5439))
* Introduce API Tokens with `cluster_permissions` and `index_permissions` directly associated with the token ([#5443](https://github.com/opensearch-project/security/pull/5443))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change log needs to be cleaned up to only add this entry.

if (!issueApiTokenAllowed()) {
throw new OpenSearchSecurityException("Api token generation is not enabled.");
}
final User user = threadPool.getThreadContext().getTransient(ConfigConstants.OPENDISTRO_SECURITY_USER);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't seem like this is used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its used in the log message in the catch block down below

throw new RuntimeException(ex);
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should do this before we merge this.

cwperks added 9 commits March 16, 2026 21:15
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC] Support for API Keys in OpenSearch Security Plugin

5 participants