feat/add arm64 support #613

Volatus · 2023-03-24T23:43:31Z

This PR adds the Dockerfile to make linux/arm64 and darwin/arm64 builds easier/possible.

closes #603
closes #505

nyurik · 2023-03-24T23:45:39Z

Thanks! What's the debian-based final container size compared to alpine?

Volatus · 2023-03-24T23:47:19Z

Thanks! What's the debian-based container size compared to alpine?

On darwin/arm64 (which realistically is linux/arm64 as Docker runs on a VM on Macs) with the latest version of Docker Desktop, the final image size seems to be 98MB.

nyurik · 2023-03-24T23:51:23Z

We need to check the size diff for i64

Volatus · 2023-03-24T23:53:49Z

We need to check the size diff for i64

Not sure I quite got this. Are we talking about potential size differences of signed integers or the image size for x86 with the Debian based image?

nyurik · 2023-03-25T00:15:24Z

lol, no, i meant x86-ia64 (amd64)

nyurik · 2023-03-25T00:25:41Z

also, you may want to rebase on #614

nyurik · 2023-03-25T00:58:18Z

Ok, i just checked. When running just docker-build with the old dockerfile (using the #614 for speed up):

> just docker-build
...
> docker images
REPOSITORY                                TAG             IMAGE ID       CREATED          SIZE
ghcr.io/maplibre/martin                   latest          8b3e3e02fc34   22 minutes ago   31.1MB

with the proposed docker image (same cargo patch), the result is

REPOSITORY                                TAG             IMAGE ID       CREATED          SIZE
ghcr.io/maplibre/martin                   latest          aadb127bb44d   31 seconds ago   104MB

I don't think growing the image size by over 3x just to make mac docker images is a good proposition because the vast majority of these images are used on linux servers (this might change in the future).

Volatus · 2023-03-25T10:30:23Z

Ok, i just checked. When running just docker-build with the old dockerfile (using the #614 for speed up):
> just docker-build
...
> docker images
REPOSITORY                                TAG             IMAGE ID       CREATED          SIZE
ghcr.io/maplibre/martin                   latest          8b3e3e02fc34   22 minutes ago   31.1MB
with the proposed docker image (same cargo patch), the result is
REPOSITORY                                TAG             IMAGE ID       CREATED          SIZE
ghcr.io/maplibre/martin                   latest          aadb127bb44d   31 seconds ago   104MB
I don't think growing the image size by over 3x just to make mac docker images is a good proposition because the vast majority of these images are used on linux servers (this might change in the future).

The size may be 3x difference, but numerically, is 100MB ever a dealbreaker over 30MB for users where you can assume their average image size is > 300MB? (We use martin and our images are way above that usually)

The other thing is, this PR is not about fixing Mac images (in fact there are no such thing as Mac images, they're all linux, just different CPU arch depending on the machine), it's about fixing arm64 as a whole.

If you try to build the old image on linux/arm64, it'll fail due to an issue with musl and glibc differences on arm64 intrinsics. I can attempt to bring the image size down a bit, but I doubt it's worth the effort. I'd say that solid images on Dockerhub are usually below the 200MB mark and this change still keeps us there.

Volatus · 2023-03-25T10:30:39Z

also, you may want to rebase on #614

Can do.

Volatus · 2023-03-25T10:50:31Z

@nyurik Just rebased on top of 8ef8953

Volatus · 2023-03-25T13:32:25Z

@nyurik I was able to cut 10MB by stripping the symbols release binary.

nyurik · 2023-03-25T17:10:53Z

Sorry, I meant arm64 for linux. I don't think shaving the 3MB and loosing all symbols is a good path. I would much rather have proper stack traces with symbols if something happens, rather than having hard-to-parse stacktraces -- those 3MB actually add value. What I am concerned with is that we want to add 70MB to an image without any benefit to the existing user base. 70MB might not sound like a lot, but this is a waste that I would rather avoid unless absolutely must have.

I propose we figure out how to build with musl/libc on arm64 - it shouldn't be THAT difficult I would think. If we can't, lets just create a new arm64.Dockerfile that we will use for arm64 builds.

nyurik · 2023-03-25T18:09:19Z

Regardless - I don't think that's the issue - there seem to be some issue with dockerx and manifests - and that's the same thing I saw before with @stepankuzmin 's PR

Volatus · 2023-03-25T20:24:42Z

Regardless - I don't think that's the issue - there seem to be some issue with dockerx and manifests - and that's the same thing I saw before with @stepankuzmin 's PR

That is a known issue - docker/buildx#59

It can be remedied by building the ARM image as a separate step. I could work on that, as long as there is some guarantee that the working solution would be merged. Wouldn't want to work on something that maintainers don't want to go forward with.

nyurik · 2023-03-25T21:22:47Z

That I'm sure we will do, just possibly as arm64 specific docker file so to keep the image size low for others

Volatus · 2023-03-25T21:25:42Z

That I'm sure we will do, just possibly as arm64 specific docker file so to keep the image size low for others

Got it, then I think I should go forward with arm64 specific Dockerfile and leave the current one as it is then.

Signed-off-by: Ismayil Mirzali <[email protected]>

Volatus · 2023-03-26T16:42:46Z

@nyurik Seems to build fine, although slow, which is expected with emulation. For the time being, we can't run the DB tests on the ARM image yet because of unavailability of an ARM image for PostGIS. I'm looking into submitting a patch for that down the line. I think for the time being, we can assume that the DB tests that passed on x86 would also pass on arm64.

nyurik

looks great, thanks!

nyurik · 2023-03-28T16:37:55Z

@Volatus or @stepankuzmin could you verify that the arm64 image is published properly under the main tag?

Volatus · 2023-03-28T16:40:34Z

@Volatus or @stepankuzmin could you verify that the arm64 image is published properly under the main tag?

@nyurik
The image seems to be there but the manifest list entry is not correct.

nyurik · 2023-03-28T16:41:52Z

that's not good :(

can it be that the publish step needs to be modified as well somehow, to preserve the platform?

stepankuzmin · 2023-03-28T16:42:09Z

Yep, should fix the manifest

Unable to find image 'ghcr.io/maplibre/martin:latest' locally
latest: Pulling from maplibre/martin
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.

Volatus · 2023-03-28T16:43:21Z

that's not good :(

can it be that the publish step needs to be modified as well somehow, to preserve the platform?

Maybe try adding the platforms option to the push step as well?

nyurik · 2023-03-28T16:50:39Z

Hmm, probably. I pushed it to main (no point in going via PRs because PRs don't trigger that step anyway)... will see.

nyurik · 2023-03-28T19:04:16Z

Hmm, it seems the push step for some reason re-builds the docker image -- https://github.com/maplibre/martin/actions/runs/4545573925/jobs/8013094651

This is not good for several reasons - most importantly because it seems it uses a different dockerfile in that step... Any thoughts of why that might be?

nyurik · 2023-03-29T01:52:45Z

@Volatus @stepankuzmin please retest - i think i fixed it, but no idea if the metadata is correct

nyurik · 2023-03-29T01:54:46Z

Weirdly enough, there are two architectures listed: arm64 and unknown. Should the x86-amd64 specify its metadata in some other way? https://github.com/maplibre/martin/pkgs/container/martin/81143769?tag=main

nyurik · 2023-03-29T01:59:48Z

And one more interesting tidbit: the image tagged as arm64 is actually amd64! And I wasn't able to download the unknown/unknown image, so i have no idea what that one is... Any ideas how to fix this? I would hate to revert this PR.

❯ docker pull --platform linux/arm64 ghcr.io/maplibre/martin:main
main: Pulling from maplibre/martin
fcdb9667c46b: Pull complete
bc959a36ad3b: Pull complete
Digest: sha256:248ef4abb68d3fe2dc1af2068984850fad48bc3d500af67bbfb9b3485620f8c9
Status: Downloaded newer image for ghcr.io/maplibre/martin:main
WARNING: image with reference ghcr.io/maplibre/martin was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64
ghcr.io/maplibre/martin:main

nyurik · 2023-03-29T04:11:59Z

I did receive a reply on stackoverflow about this https://stackoverflow.com/questions/75872905/building-both-amd64-and-arm64-with-github-workflows-from-different-files/75873049#75873049

Volatus · 2023-03-29T08:42:45Z

I think if both of them have their own manifest step, it should end up working.

Volatus · 2023-03-29T10:41:34Z

@nyurik It's surely not the case the issue is due to order right?

martin/.github/workflows/docker.yml

Lines 21 to 25 in eae422a

    
           include: 
        
             - platform: linux/amd64 
        
               file: Dockerfile 
        
             - platform: linux/arm64 
        
               file: arm64.Dockerfile

martin/.github/workflows/docker.yml

Lines 42 to 43 in eae422a

    
           with: 
        
             platforms: linux/arm64,linux/amd64

nyurik · 2023-03-29T15:47:22Z

@Volatus definitly not because i added the QEMU configuration after I saw the issue. Before this, QEMU was configured with ALL

This reverts commit c358ec5, as well as any other related changes to the docker github action. It is clearly not working, while also not allowing us to build proper releases quicker. Several ways to fix: * Use 3rd party CI service to just build multi-platform docker images * Use cross-compilation wiht github actions * (???)

This reverts commit c358ec5, as well as any other related changes to the docker github action. It is clearly not working, while also not allowing us to build proper releases quicker. Several ways to fix: * Use 3rd party CI service to just build multi-platform docker images * Use cross-compilation wiht github actions * (???) See also #655, #603 and #505

Volatus force-pushed the feat/add-arm64-support branch 2 times, most recently from 7af0101 to 64146d2 Compare March 25, 2023 10:48

Volatus force-pushed the feat/add-arm64-support branch 2 times, most recently from e4b7251 to 481eee6 Compare March 26, 2023 13:28

feat: add CI build for ARM64 Docker image

60bef90

Signed-off-by: Ismayil Mirzali <[email protected]>

Volatus force-pushed the feat/add-arm64-support branch from 481eee6 to 60bef90 Compare March 26, 2023 15:58

nyurik added 3 commits March 28, 2023 00:56

make ci workflow match proposed one

5099c5a

remove duplicate workflow

56b0b86

limit docker testing to amd64

3710efb

nyurik enabled auto-merge (squash) March 28, 2023 06:04

nyurik approved these changes Mar 28, 2023

View reviewed changes

nyurik merged commit c358ec5 into maplibre:main Mar 28, 2023

Volatus deleted the feat/add-arm64-support branch March 28, 2023 08:16

nyurik mentioned this pull request May 9, 2023

Docker image for arm64 platform #655

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat/add arm64 support #613

feat/add arm64 support #613

Volatus commented Mar 24, 2023 •

edited

Loading

nyurik commented Mar 24, 2023 •

edited

Loading

Volatus commented Mar 24, 2023 •

edited

Loading

nyurik commented Mar 24, 2023

Volatus commented Mar 24, 2023

nyurik commented Mar 25, 2023

nyurik commented Mar 25, 2023

nyurik commented Mar 25, 2023 •

edited

Loading

Volatus commented Mar 25, 2023 •

edited

Loading

Volatus commented Mar 25, 2023

Volatus commented Mar 25, 2023 •

edited

Loading

Volatus commented Mar 25, 2023

nyurik commented Mar 25, 2023

nyurik commented Mar 25, 2023

Volatus commented Mar 25, 2023

nyurik commented Mar 25, 2023

Volatus commented Mar 25, 2023

Volatus commented Mar 26, 2023 •

edited

Loading

nyurik left a comment

nyurik commented Mar 28, 2023

Volatus commented Mar 28, 2023

nyurik commented Mar 28, 2023

stepankuzmin commented Mar 28, 2023

Volatus commented Mar 28, 2023

nyurik commented Mar 28, 2023

nyurik commented Mar 28, 2023

nyurik commented Mar 29, 2023

nyurik commented Mar 29, 2023

nyurik commented Mar 29, 2023

nyurik commented Mar 29, 2023

Volatus commented Mar 29, 2023

Volatus commented Mar 29, 2023 •

edited

Loading

nyurik commented Mar 29, 2023

feat/add arm64 support #613

feat/add arm64 support #613

Conversation

Volatus commented Mar 24, 2023 • edited Loading

nyurik commented Mar 24, 2023 • edited Loading

Volatus commented Mar 24, 2023 • edited Loading

nyurik commented Mar 24, 2023

Volatus commented Mar 24, 2023

nyurik commented Mar 25, 2023

nyurik commented Mar 25, 2023

nyurik commented Mar 25, 2023 • edited Loading

Volatus commented Mar 25, 2023 • edited Loading

Volatus commented Mar 25, 2023

Volatus commented Mar 25, 2023 • edited Loading

Volatus commented Mar 25, 2023

nyurik commented Mar 25, 2023

nyurik commented Mar 25, 2023

Volatus commented Mar 25, 2023

nyurik commented Mar 25, 2023

Volatus commented Mar 25, 2023

Volatus commented Mar 26, 2023 • edited Loading

nyurik left a comment

Choose a reason for hiding this comment

nyurik commented Mar 28, 2023

Volatus commented Mar 28, 2023

nyurik commented Mar 28, 2023

stepankuzmin commented Mar 28, 2023

Volatus commented Mar 28, 2023

nyurik commented Mar 28, 2023

nyurik commented Mar 28, 2023

nyurik commented Mar 29, 2023

nyurik commented Mar 29, 2023

nyurik commented Mar 29, 2023

nyurik commented Mar 29, 2023

Volatus commented Mar 29, 2023

Volatus commented Mar 29, 2023 • edited Loading

nyurik commented Mar 29, 2023

Volatus commented Mar 24, 2023 •

edited

Loading

nyurik commented Mar 24, 2023 •

edited

Loading

Volatus commented Mar 24, 2023 •

edited

Loading

nyurik commented Mar 25, 2023 •

edited

Loading

Volatus commented Mar 25, 2023 •

edited

Loading

Volatus commented Mar 25, 2023 •

edited

Loading

Volatus commented Mar 26, 2023 •

edited

Loading

Volatus commented Mar 29, 2023 •

edited

Loading