ARM images for AWS public registry

[daz10000]

Bug report

There's a chance I'm doing something boneheaded here, but I am trying to pull the latest seqera-labs AWS batch docker images from public.ecr.aws/seqera-labs/nextflow:latest from an AWS c6g linux/arm64/8 image, and getting linux/amd64 images.

Expected behavior and actual behavior

The seqera-labs gallery suggests that arm images are available. From an arm64 machine, I am invoking this build with the Dockerfile below

docker build --platform linux/arm64 .

I would expect that to select the arm images (I couldn't see anything from the gallery that indicated architecture) but I'm assuming the docker build grabs the right one.

ARG VERSION=latest
FROM public.ecr.aws/seqera-labs/nextflow:${VERSION} AS build


RUN yum update -y \
 && yum install -y \
    unzip \
 && yum clean -y all
RUN rm -rf /var/cache/yum

# install awscli v2
RUN curl -s "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "/tmp/awscliv2.zip" \
 && unzip -q /tmp/awscliv2.zip -d /tmp \
 && /tmp/aws/install -b /usr/bin \
 && rm -rf /tmp/aws*

# install a custom entrypoint script that handles being run within an AWS Batch Job
COPY nextflow.aws.sh /opt/bin/nextflow.aws.sh
RUN chmod +x /opt/bin/nextflow.aws.sh

WORKDIR /opt/work
ENTRYPOINT ["/opt/bin/nextflow.aws.sh"]

When I don't explicitly add the target, I get an architecture error - it clearly pulled the wrong version.

$ docker build .
Sending build context to Docker daemon  7.168kB
Step 1/9 : ARG VERSION=latest
Step 2/9 : FROM public.ecr.aws/seqera-labs/nextflow:${VERSION} AS build
latest: Pulling from seqera-labs/nextflow
8be3d01330d7: Pull complete
05200b11ae6e: Pull complete
4d4e2f332acf: Pull complete
f29f4675beae: Pull complete
1f0835b691b9: Pull complete
85af6e7a5c27: Pull complete
4f8bff6a50cd: Pull complete
Digest: sha256:01bdca61259a1a431c9d8534c0bb8353d413cce8f9638bb8d4e7b854cb5ce43c
Status: Downloaded newer image for public.ecr.aws/seqera-labs/nextflow:latest
 ---> a1c176ab3423
Step 3/9 : RUN yum update -y  && yum install -y     unzip  && yum clean -y all
 ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in d0d491444ad2
exec /bin/sh: exec format error
The command '/bin/sh -c yum update -y  && yum install -y     unzip  && yum clean -y all' returned a non-zero code: 1

When I force the architecture explicitly, docker warns me

WARNING: Pulled image with specified platform (linux/arm64), but the resulting image's configured platform (linux/amd64) does not match.
This is most likely caused by a bug in the build system that created the fetched image (public.ecr.aws/seqera-labs/nextflow:23.04.3).
Please notify the image author to correct the configuration.

Steps to reproduce the problem

On an arm64 machine, using the Dockerfile above, try

docker build --platform linux/arm64 .

Program output

See above

Environment

• Linux aws c6g instance
• 6.1.34-59.116.amzn2023.aarch64 # 1 SMP Thu Jun 29 18:11:49 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
• Docker version 20.10.23, build 7155243

Additional context

Hoping it's something dumb I've done. I'd be happy to try to build the image from scratch - looking at the source right now to figure out how it was probably built.

[pditommaso]

@bentsherman can you please have a look

[daz10000]

I managed to find the build infrastructure and build my own Docker image, so this isn't blocking me. My best guess is that something in the CI/CD chain doesn't have the right docker drivers to buildx an arm image, so it's just building two amd images, but I might be wrong. When I was doing the reverse (trying to build the arm image on an arm machine, only the build-arm target would work, the regular build target would fail). Again, could be something in how I'm downloading the images, but my understanding is that the tagging should sort that out. One thing I noticed is that the Dockerfile itself hard codes the TARGETPLATFORM as amd, but the Makefile should be overriding that. Maybe that's not working.

[bentsherman]

Hi @daz10000 , can you try again? I think Paolo fixed something with the ECR images, and I just tried your example and it pulled the image successfully.

[pditommaso]

nope. I've fixed the container for the rnaseq-nf pipeline, not this

[bentsherman]

Hmm, it probably worked for me because I pulled from my laptop, need to try from an ARM instance instead