Why token is generated when pull the pulp container image where token auth is in disabled?

[mchithu]

Version
pulpcore: 3.80.1
container: 2.25.1
pulp-rpm: 3.29.0

Describe the bug
Strangely, we faced an 403 authentication issue when try pull the container image using SHA digest and not when using tag value.

Why token is generated when token authentication is disabled in settings.py?
What is the role REDIS Cache TTL in the Pulp container image pull?
Why 403 Authentication issue occurs only when we try to pull the container image using the SHA digests and why not when we pull an image using the tag value?
How the data flow is when we pull a container image? When the container is image publicly made available is only the pulp-content task is supply the docker container image layers or pulp-api is also involved in that request? Likely what is flow when container image pull based on authentication? If pulp-api handles the authentication, after validation of the user how it passes the authorization message to pulp-content to supply the private container image.

To Reproduce

Expected behavior

Additional context

[gerrod3]

@mchithu The data flow for pulp_container goes something like this:

podman pull -> pulp_container registry api (pulp-api) -> authentication (basic or token) -> find the requested image (either through digest or tag) -> if found redirect to the content app using redirect content guard -> content app returns s3/azure redirect link or streams manifest if file storage.

Now the way we pass auth from the pulp-api to the pulp-content is through the redirect content guard which generates an authenticated link (with a 60 minute expiration). When using redis caching we cache both the pulp-api request and the pulp-content request. So in the above data-flow after the first authentication we would check the cache and return the saved content app redirect link if present. If the cache TTL is greater than 60 minutes then you will get auth errors on your pulls. The client will then request the content app and we'll check once again in our cache to see if we saved the redirect link (if using S3/Azure). And if you set up your S3/Azure storage to use auth links and the cache's TTL is greater than their valid expiration then you can again get auth errors on the pull.

I was not able to reproduce generating tokens with the setting disabled. As for why you were seeing the issue only when pulling by digest and not tag, that I do not know.