Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker_client: Handle "invalid_scope" errors #1079

Closed
wants to merge 1 commit into from

Conversation

rhafer
Copy link
Contributor

@rhafer rhafer commented Nov 11, 2020

podman 2.1.1 (which uses containers/images 5.7.0) fails to search some registries. E.g.

podman --log-level=debug search registry.suse.com/

gives:

[...]
DEBU[0000] trying to talk to v2 search endpoint         
DEBU[0000] GET https://registry.suse.com/v2/            
DEBU[0000] Ping https://registry.suse.com/v2/ status 401 
DEBU[0000] GET https://registry.suse.com/auth?service=SUSE+Linux+Docker+Registry 
DEBU[0000] Increasing token expiration to: 60 seconds   
DEBU[0000] GET https://registry.suse.com/v2/_catalog    
ERRO[0000] error getting search results from v2 endpoint "registry.suse.com": unable to retrieve auth token: invalid username/password: 
denied: requested access to the resource is denied

Looking into the details of the requests and response and trying to repeat it with curl I can see that the request to /v2/_catalog failes with Status 401 and the www-authenticate containing this error:

< www-authenticate: Bearer realm="https://registry.suse.com/auth",service="SUSE Linux Docker Registry",scope="registry:catalog:*",error="insufficient_scope"

This can be fixed by simply fetching a new token with the scope value set to the value return in that header.

This PR tries to implement a fix for the above behavior by checking for the error and retrying the request with and updated token.

Copy link
Collaborator

@mtrmac mtrmac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

There’s an immediate concurrency concern. Other than that, based on a very quick initial look, this seems useful and reasonable at a first glance.

(Note that Parameters["scope"] is not used at all, because Docker has historically ignored it, and some implementations just don’t return useful values. Using that only on a failure where the previous code would give up is almost certainly fine, but it will be necessary for me to return to the old research.)

@@ -627,6 +660,11 @@ func (c *dockerClient) getBearerTokenOAuth2(ctx context.Context, challenge chall
if service, ok := challenge.Parameters["service"]; ok && service != "" {
params.Add("service", service)
}

if scope, ok := challenge.Parameters["scope"]; ok && scope != "" {
params.Add("scope", scope)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: Should this be added or replace the original?

// check for that and update the client challenges to retry with requesting a new token
if retry, challenges := needsRetryWithUpdatedChallenges(err, resp); retry {
logrus.Debug("Detected insufficent_scope error, retrying request with updated challenges")
c.challenges = challenges
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dockerClient, once initialized, can be used concurrently (HasThreadSafe…), so overwriting a property which is intentionally in the “… and never change afterwards” section of dockerClient is not safe. (This is also why extraScope is a parameter and not a property.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok. Must have overlooked that :(. I had initially thought to put the updated scope into the extraScope argument, but as that would have required to rework the authScope struct a bit (as the current code always assumes the resourcetype part of it to be repository I went of the above approach in my first try. I guess I need to rethink that again :).

What do you think, does it make sense to pass the updated scope via the extraScope argument?

@rhatdan
Copy link
Member

rhatdan commented Mar 23, 2021

@mtrmac @vrothberg PTAL
@rhafer Probably could use a rebase.

By default docker_client just uses the auth challenges from the /v2/
ping request to request a Bearer Token. For some requests (e.g. for
/v2/_catalog on some registries) this might not be sufficient and return a
a HTTP Unauthorized Error with the "www-authenticate" header including
an "insufficient_scope" error. In that case the client will now retry
the request and fetch a new token with updated challenges to have the
"scope" matching for what the endpoint needs.

Signed-off-by: Ralf Haferkamp <[email protected]>
Signed-off-by: Ralf Haferkamp <[email protected]>
@rhafer
Copy link
Contributor Author

rhafer commented Feb 23, 2022

This would need another rebase. But I am not able to spend any time on this. So I'll just close it.

@mtrmac
Copy link
Collaborator

mtrmac commented Feb 23, 2022

Sorry about that, not getting back to the PR is totally our fault. I have at least filed #1478 so that we track this action item.

@josegomezr
Copy link

@rhafer Could I take over this code? I have some free time I'd like to invest in this.

@rhafer
Copy link
Contributor Author

rhafer commented Mar 1, 2022

@josegomezr Sure, go ahead. As stated I won't be able to spent any time on this. (Neither does this bug affect me personally anymore)

@rhatdan
Copy link
Member

rhatdan commented May 2, 2022

@josegomezr Did you ever work on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants