Skip to content

Improve error handling in Quay client HTTP requests #1057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fao89 wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Improve error handling in Quay client HTTP requests #1057

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dbd5a80
Improve error handling in Quay client HTTP requests
fao89 Dec 10, 2025
48173ac
Fix Quay.io API filter parameter causing 403 Forbidden errors
fao89 Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions cincinnati/src/plugins/internal/metadata_fetch_quay.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,15 +123,19 @@ impl InternalPlugin for QuayMetadataFetchPlugin {
self.label_filter.clone(),
);

let quay_labels = client
let all_labels = client
.get_labels(
repo.clone(),
manifestref.clone(),
Some(label_filter.clone()),
None, // Remove filter to avoid 403 Forbidden - filter client-side instead
)
.await?
.await?;
Comment on lines +126 to +132
Copy link

@coderabbitai coderabbitai bot Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

rg -n -C3 'get_labels|label.*filter' --type=rust -g '!target/**'

Repository: openshift/cincinnati

Length of output: 184

🌐 Web query:

Quay.io API label filtering 403 Forbidden error

💡 Result:

Likely causes and quick checks for a 403 when filtering by labels on the Quay API

  1. Permission/scope issue (most common)
  • The API call requires repo:read (or repo:write for label changes). Ensure the token/robot account has the needed repository permissions or is a team member of the org. Test with a token that you know has repo:read. [1][2]
  1. Using the wrong account type
  • Robot accounts must be used for some automated operations (and must have read access to the repo). If using a user token, confirm the user is member of the org and has repo access. [1]
  1. Wrong endpoint/parameters or path
  • The manifest/labels endpoints include repository path and manifest digest in the path; incorrectly-formed paths or query parameters can return 403/401. Verify you’re calling the documented endpoint (e.g. GET /api/v1/repository/{repository}/manifest/{manifestref}/labels with optional filter). [2]
  1. Authentication header or token not sent
  • Confirm Authorization: Bearer is present and not being removed by proxies or client libs. Also check you’re not accidentally using a cookie/session token where an OAuth token is required. [1]
  1. Registry configuration / proxy or signature mismatch
  • If you see 403 with XML or “SignatureDoesNotMatch” in body, it can be a backend S3/signing issue or proxy that changes requests. Check registry logs and any reverse proxy in path. (Quay docs and support notes mention S3/signature and proxy issues producing 403s.) [1][3]

How to debug (quick steps)

  • Reproduce with curl (explicit header) and full verbose output:
    curl -v -H "Authorization: Bearer " "https://quay.io/api/v1/repository/namespace/name/manifest//labels?filter=keyprefix"
  • Try a token you know has repo:read to confirm permission vs token problem.
  • Check Quay server logs for the request and reason (auth/acl check or upstream storage error).
  • If using a robot, confirm it’s enabled and has read on that repo (and that you didn’t use a user token that lacks org membership). [1][2]

If you want, paste (redacted) the curl command and full response (headers + body) or the exact endpoint and token type you’re using and I’ll point to the specific cause and fix.

Sources

  • Quay docs — “I’m authorized but I’m still getting 403s” / common auth failures. [1]
  • Red Hat / Quay API guide — manifest/labels endpoints and required authorizations. [2]
  • Quay/Red Hat issues & support notes re: 403 from storage/signature problems. [3]

References
[1] Quay docs — auth failure guidance.
[2] Red Hat Quay API Guide — manifest/labels endpoints and permissions.
[3] Red Hat/Quay support notes on 403s (S3/signature/proxy).

Investigate and fix the root cause of the 403 Forbidden error instead of working around it.

The change from server-side to client-side filtering suggests an underlying authentication or permissions issue with the Quay API token/robot account. Based on Quay API documentation, 403 errors on filtered label queries typically indicate:

  • The token lacks repo:read permissions on the repository
  • The robot account or user is not properly configured
  • Incorrect API endpoint path or authentication headers

Client-side filtering is a workaround that masks the actual problem. Verify the credentials have appropriate permissions and that the API call is correctly formed before removing the server-side filter. This also has performance implications if filtering many labels.

🤖 Prompt for AI Agents 
In cincinnati/src/plugins/internal/metadata_fetch_quay.rs around lines 126 to
132, stop masking a 403 by removing the server-side filter; investigate and fix
the root cause so the Quay API can accept filtered queries. Check the
credentials used by the client: verify the token/robot account has repo:read
(and other required) permissions on the target repo, confirm the robot account
is correctly provisioned, and ensure the request uses the correct Quay API
endpoint and authentication headers. Add explicit handling/logging for 403
responses to surface permission errors (include API path, token identifier, and
response body) and restore the server-side filter once permissions/endpoint/auth
are corrected to avoid client-side filtering and the associated performance
cost.


// Filter client-side for labels matching the expected prefix
let quay_labels = all_labels
.into_iter()
.map(Into::into)
.filter(|(key, _): &(String, String)| key.starts_with(&label_filter))
.collect::<Vec<(String, String)>>();

labels_with_releaseinfo.push((quay_labels, (release_id, release_version)));
Expand Down
7 changes: 7 additions & 0 deletions quay/src/v1/manifest.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,13 @@ impl Client {
})?;

let resp = req.send().await?;

// Check if the response was successful
if !resp.status().is_success() {
let status = resp.status();
return Err(anyhow::anyhow!("Request failed with status {}", status));
}

let json = resp.json::<Labels>().await?;

Ok(json.labels)
Expand Down
8 changes: 8 additions & 0 deletions quay/src/v1/tag.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,14 @@ impl Client {
.query(&[("onlyActiveTags", actives_only)]);

let resp = req.send().await?;

// Check if the response was successful
if !resp.status().is_success() {
let status = resp.status();
yield Err(anyhow::anyhow!("Request failed with status {}", status));
return;
}

let paginated_tags = resp.json::<PaginatedTags>().await?.tags;
for tag in paginated_tags {
yield Ok(tag);
Expand Down
15 changes: 10 additions & 5 deletions quay/tests/net/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,14 +60,19 @@ fn test_public_get_labels() {
let fetch_labels = client.get_labels(
repo.to_string(),
digest,
Some("io.openshift.upgrades.graph".to_string()),
None, // Remove filter to avoid 403 Forbidden - filter client-side instead
);
let labels = rt.block_on(fetch_labels).unwrap();

// Filter client-side for labels matching the expected prefix
let filtered_labels: Vec<(String, String)> = labels
.into_iter()
.map(Into::into)
.filter(|(key, _)| key.starts_with("io.openshift.upgrades.graph"))
.collect();

assert_eq!(
labels
.into_iter()
.map(Into::into)
.collect::<Vec<(String, String)>>(),
filtered_labels,
vec![(
"io.openshift.upgrades.graph.previous.remove".to_string(),
"0.0.0".to_string()
Expand Down