docker-credential-gcr: Workload Identity Federation

[hermanbanken]

The library kaniko is using for GCR authentication is updated to support Workload Identity Federation (type = external_account). See issue GoogleCloudPlatform/docker-credential-gcr#95
. Unfortunately Kaniko is using an older version so we have
the following issue.

Actual behavior

error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "eu.gcr.io/$PROJECT/repo": resolving authorization for eu.gcr.io failed: error getting credentials - err: exit status 1, out: `docker-credential-gcr/helper: could not retrieve GCR's access token: google: error getting credentials using GOOGLE_APPLICATION_CREDENTIALS environment variable: unknown credential type: "external_account"`

Expected behavior
Working pull/push/builds.

To Reproduce
Steps to reproduce the behavior:

1. Create a GitHub Action workflow.
2. Use https://github.com/google-github-actions/auth to prepare authentication. This sets GOOGLE_APPLICATION_CREDENTIALS to a file with the format as ducmented in Does this support Identity Federation from external accounts? GoogleCloudPlatform/docker-credential-gcr#95.
3. Use Kaniko to build an image

Additional Information

• Kaniko Image: latest (sha256:0fae223f496525e31226cde473ec77ed15abfa8cbabff5ff5bf1c5268355bbb0).

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile

Notes

I have a hard time figuring out if this should work: whether or not the change is in or out on GitHub-master and on GCR-latest.

This line must be present for it to work: https://cs.opensource.google/go/x/oauth2/+/master:google/google.go;l=97;drc=ba495a64dcb59dd28baeda5ded53504191dea54f
it was implemented in this PR https://go-review.googlesource.com/c/oauth2/+/276312, merged on Tue Jan 12 19:45:47 2021.

Kaniko uses https://github.com/GoogleCloudPlatform/docker-credential-gcr/commits/4cdd60d0f2d8a69bc70933f4d7718f9c4e956ff8

RUN go install github.com/GoogleCloudPlatform/docker-credential-gcr@4cdd60d0f2d8a69bc70933f4d7718f9c4e956ff8

which is using golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f.

Weirdly enough, it is not working in latest. Where does one quickly find what is in latest on GCR?

Update: running docker-credential-gcr version inside the debug image reports Google Container Registry Docker credential helper 2.0.1.

Only version 2.1.0 supports external_account, which was added when they moved to Go Modules.

[hermanbanken]

Actually, version 4cdd60d0f2d8a69bc70933f4d7718f9c4e956ff8 is 2.1.0. So the main branch is including support already, it just needs to be released!

[hermanbanken]

Tested with Kaniko gcr.io/kaniko-project/executor:76624697df879f7c3e3348f22b8c986071af4835-debug which is the current main version:

WARN[0000] Failed to setup Google env authenticator, ignoring: google: error getting credentials using GOOGLE_APPLICATION_CREDENTIALS environment variable: unknown credential type: "external_account" 

still happens.

Caused by

kaniko/pkg/creds/creds_linux.go

Lines 44 to 46 in 9861215

	gauth, err := google.NewEnvAuthenticator()
	if err != nil {
	logrus.Warnf("Failed to setup Google env authenticator, ignoring: %v", err)

, due to the vendoring of https://github.com/GoogleContainerTools/kaniko/blob/main/vendor/golang.org/x/oauth2/google/default.go which is pretty old.

[imjasonh]

That's calling through https://github.com/google/go-containerregistry/blob/19c2b639fab1/pkg/v1/google/auth.go#L50 (which is indeed pretty old!)

#1885 bumps both go-containerregistry to main, and x/oauth2 to d3ed0bb246c8 which includes the external_account support.

[hermanbanken]

creds.json:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
  },
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/SERVICE_ACCOUNT_EMAIL:generateAccessToken"
}

Dockerfile:

FROM alpine

test.sh:

GOOGLE_APPLICATION_CREDENTIALS=/pwd/creds.json /kaniko/executor --destination=gcr.io/somewhere/overthe/rainbow --dockerfile=/pwd/Dockerfile -v=debug

Quick test:

# tag 1.6.0 / debug
docker run -it -v $PWD:/pwd --entrypoint sh gcr.io/kaniko-project/executor:a0c96b4edcaeee437243484f5880833518d06551-debug /pwd/test.sh -v=debug
# Output: 
# error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "gcr.io/somewhere/overthe/rainbow": resolving authorization for gcr.io failed: error getting credentials - err: exit status 1, out: `docker-credential-gcr/helper: could not retrieve GCR's access token: google: error getting credentials using
# GOOGLE_APPLICATION_CREDENTIALS environment variable: unknown credential type: "external_account"`

# 1.7.0
docker run -it -v $PWD:/pwd --entrypoint sh gcr.io/kaniko-project/executor:28a6bdce78a5f5cb09cc4cc650595721554eef6b-debug /pwd/test.sh -v=debug
# Output, as expected with fake credential file: 
# dial tcp 169.254.169.254:80: connect: connection refused

# main branch after PR 1890
docker run -it -v $PWD:/pwd --entrypoint sh gcr.io/kaniko-project/executor:6bbd8a331cbd361fe6dadcc2089adc4eb0e8ca08-debug /pwd/test.sh -v=debug
# Output, as expected with fake credential file: 
# dial tcp 169.254.169.254:80: connect: connection refused