kaniko issue with multiple AWS ECR registries using aws profiles

[papillon88]

Refernce link - https://wadehuang36.medium.com/use-aws-ecr-credential-helper-with-multiple-accounts-4c38b7ee37d

First create the kaniko image -
dockerfile to create a custom kaniko image from the upstream kaniko image

FROM gcr.io/kaniko-project/executor:debug AS kaniko

FROM our-custom-openjdk8-image-with-aws:latest

COPY --from=kaniko /kaniko/executor /kaniko/executor

COPY --from=kaniko /kaniko/docker-credential-ecr-login /kaniko/docker-credential-ecr-login

COPY --from=kaniko /kaniko/warmer /kaniko/warmer

COPY --from=kaniko /kaniko/ssl/certs/ca-certificates.crt /kaniko/ssl/certs/ca-certificates.crt

COPY --from=kaniko /kaniko/.docker /kaniko/.docker

ENV PATH $PATH:/usr/local/bin:/kaniko

ENV DOCKER_CONFIG /kaniko/.docker/

Now run the image and exec into it -
docker run -it --entrypoint=/busybox/sh my-custom-kaniko-image:latest

Now inside the container :
Now below is the config.json -
/kaniko/.docker/config.json file

{
    "credHelpers": {
        "1234.dkr.ecr.region.amazonaws.com": "ecr-login-1234",
        "5678.dkr.ecr.region.amazonaws.com": "ecr-login-5678"
    }
}

Below is the aws credential file -
~/.aws/credentials file

cat ~/.aws/credentials
[account1]
aws_access_key_id = ABCD
aws_secret_access_key = WXYZ
[account2]
aws_access_key_id = EFGH
aws_secret_access_key = PQRS

Now I created 2 shell scripts under /kaniko dir
script #1 - docker-credential-ecr-login-1234.sh

#contents
#!/bin/sh
export AWS_PROFILE=account1
docker-credential-ecr-login "$@"

script #2 - docker-credential-ecr-login-5678.sh

#contents
#!/bin/sh
export AWS_PROFILE=account2
docker-credential-ecr-login "$@"

I now run

/kaniko/executor --destination "1234.dkr.ecr.us-east-1.amazonaws.com/test-reporsitory:cbs-sample-springboot-ms-v1-0-0-929194337" --context "empty" --dockerfile "dockerfile" -v debug

The "context" is a directory that is empty.
The "dockerfile" is below -

FROM 1234.dkr.ecr.us-east-1.amazonaws.com/test-reporsitory:cbs-sample-springboot-ms-v1-0-0-929194337
RUN apk update
RUN apk update

I get below error

....
DEBU[0000] Checking file cache                           registry=834930510981
DEBU[0000] Calling ECR.GetAuthorizationToken             registry=834930510981
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "834930510981.dkr.ecr.us-east-1.amazonaws.com/test-reporsitory:cbs-sample-springboot-ms-v1-0-0-929194337": POST https://834930510981.dkr.ecr.us-east-1.amazonaws.com/v2/test-reporsitory/blobs/uploads/: unexpected status code 401 Unauthorized: Not Authorized

Expected behavior is that I should not get Not authorized error and kaniko should be able to build and push this image.

[papillon88]

Hi, Any updates on this ?

[dev-whoan]

Hi, Any updates on this?

[Ankhas]

Hi, I have successfully achieved to do this by setting AWS environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN) instead of using the file .aws/credentials with AWS_PROFILE.

For example, in your script 1 docker-credential-ecr-login-1234.sh:

#!/bin/sh
export AWS_ACCESS_KEY_ID=$aws_access_key_id
export AWS_SECRET_ACCESS_KEY=$aws_secret_access_key
export AWS_SESSION_TOKEN=$aws_session_token
docker-credential-ecr-login "$@"