32-bit op-mode on x86_64/Linux container on Mac M1

[abmcmillan]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID: D3BC0590-445C-4BA1-B01C-769CE33AF7EE/20230406192301

Expected behavior

I am expecting to be able to run x86_64/Linux applications and a development environment on a Mac M1

Actual behavior

I get segfaults and unexpected behavior likely due to a 32-bit opmode

Information

I am trying to use Docker on my M1 MacBook to run some tools in an amd64/Linux environment. I am having issues that appear to be related to the lack of support of 64-bit instructions. I am wondering if this a limitation of the Docker Virtual machine.

Docker Version: 04.18.0 (104112)
Computer: MacBook Pro M1 / macOS (12.6.4)

For example, on my M1 MacBook I tried “docker run -it --rm --platform=linux/amd64 opensuse/leap:15.4”
then lscpu yields:
Architecture: x86_64
CPU op-mode(s): 32-bit
…

On a x86_64 Linux host, running the same command I see:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit

It seems that the lack of a 64-bit op-mode is causing segmentation faults for the application I am trying to run. Am I doing something incorrectly or is this an issue Docker on Macs or somehow related to the underlying QEMU VM?

Steps to reproduce the behavior

1. docker run -it --rm --platform=linux/amd64 opensuse/leap:15.4
2. lscpu

On a x86_64 Linux host, running the same command I see:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit

On my M1 Mac I see:
Architecture: x86_64
CPU op-mode(s): 32-bit

[sam-thibault]

Have you tried the Rosetta acceleration support from experimental settings? Its emulator might work where QEMU is falling over. (requires macos >= 13)

[abmcmillan]

Have you tried the Rosetta acceleration support from experimental settings? Its emulator might work where QEMU is falling over. (requires macos >= 13)

Good idea-- I did try this and unfortunately get the same result:

% docker run -it --rm --platform=linux/amd64 opensuse/leap:15.4

:/ # lscpu
Architecture:          x86_64
  CPU op-mode(s):      32-bit

[blazewicz]

I'm posting here a workaround found by @codeperfect, posted here: microsoft/vscode-dev-containers#1548 (comment)

This solution uses colima, an alternative docker runtime.

⚠️ Note that Docker is running under QEMU VM so expect worse performance that with native virtualization.

$ colima start --arch x86_64 --cpu 4 --memory 16
$ colima status
INFO[0000] colima is running using QEMU
INFO[0000] arch: x86_64
INFO[0000] runtime: docker
INFO[0000] mountType: sshfs
INFO[0000] socket: unix:///Users/kblazewicz/.colima/default/docker.sock
$ docker run -it --rm debian:bullseye

Now 64-bit instruction set is available from within containers:

root@66cef63d7ae7:/# lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   40 bits physical, 48 bits virtual
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
Vendor ID:                       AuthenticAMD
CPU family:                      15
Model:                           107
Model name:                      QEMU Virtual CPU version 2.5+
Stepping:                        1
CPU MHz:                         999.929
BogoMIPS:                        1999.85
Virtualization:                  AMD-V
L1d cache:                       256 KiB
L1i cache:                       256 KiB
L2 cache:                        2 MiB
L3 cache:                        16 MiB
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm rep_good nopl cpuid extd_apicid pni cx16 hype
                                 rvisor lahf_lm cmp_legacy svm 3dnowprefetch vmmcall

[3f6a]

Regarding the colima approach: QEMU is really slow, compared to Rosetta.

Are there plans to fix this in Docker? In the sense of being able to run 64 bit x86_64 Linux container (via Rosetta) on a Mac with M-series chip? Thanks!

[amirjalali1]

any update on this?
i really need 64bit version

[globant-ignacioa]

same here @amirjalali1
the Colima workaround seems to be working (I need to perform further tests), but I really need the 64-bit version

[globant-ignacioa]

same here @amirjalali1 the Colima workaround seems to be working (I need to perform further tests), but I really need the 64-bit version

I can confirm what @3f6a said: it is very, very, slow

[umang350]

Colima also has rosetta support, which is much better for 64bit x86 images in my experience. But the CPU and Memory settings can be a hit or a miss.

[dgageot]

Hi everyone, do you have an example of command that actually fails because of this? I'd love to try to reproduce.

[ooyamatakehisa]

@dgageot

docker run --platform linux/amd64 -it centos:centos7 bash

just runnning this on m1 mac ends up 32bit opmode inside the container. (can be checked bylscpu command)

Please fix this....

[robert914]

Here's a comment I put in the other issue showing the differences between an Intel and M1 using a lscpu call in a RH UBI image: docker/roadmap#384 (comment)