Docker Desktop on macOS with VirtioFS maps file permissions incorrectly

[martingd]

When using VirtioFS to map files from the macOS host filesystem to a Docker container, permissions are not mapped correctly.

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID: 63D33C3F-CAC6-403B-A459-43422AFC78B8/20230419135709

Expected behavior

When bind mounting the local macOS filesystem into a container instance I would expect files permissions (rwx) to be mapped correctly in the container. I would also expect it was possible to change permission on shared files inside the container and see that reflected in the macOS filesystem.

Actual behavior

When using osxfx (legacy) sharing, things work as expected.

When using VirtioFS sharing, I see issues with permissions:

• If a shared file is readonly in the macOS filesystem, I cannot change its permissions inside the container.
• If a shared file is writable in the macOS filesystem, I can change it to readonly inside the container. This is seen inside the container but not reflected in the permissions in the macOS filesystem.

Information

• macOS Version: Ventura, Version 13.2.1 (22D68)
• Intel chip or Apple chip: Intel
• Docker Desktop Version: 4.18.0 (104112)

Output of /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

% /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check 
[2023-04-19T13:41:29.429521000Z][com.docker.diagnose][I] set path configuration to OnHost
Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0031: does the Docker API work?
[PASS] DD0013: is the $PATH ok?
[PASS] DD0003: is the Docker CLI working?
[PASS] DD0038: is the connection to Docker working?
[PASS] DD0014: are the backend processes running?
[PASS] DD0007: is the backend responding?
[PASS] DD0008: is the native API responding?
[PASS] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[SKIP] DD0030: is the image access management authorized?
[PASS] DD0033: does the host have Internet access?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[PASS] DD0017: can a VM be started?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0031: does the Docker API work?
[PASS] DD0032: do Docker networks overlap with host IPs?
No fatal errors detected.

Steps to reproduce the behavior

Using osxfx (legacy) sharing

If “osxfx (legacy)” is selected as file sharing implementation, permissions are mapped as expected.

We run a countainer, that maps the folder “macfs" into a container. Inside the container, we create files and manipulate them.

We start the container (in this case Debian Linux) and create two files:

~/tmp/dockerchmod % docker run --rm -it -v$(pwd)/macfs:/root/macfs debian
root@5f0081ad212e:/# cd ~/macfs/
root@5f0081ad212e:~/macfs# (umask 222; touch a)
root@5f0081ad212e:~/macfs# touch b
root@5f0081ad212e:~/macfs# ls -l
total 0
-r--r--r-- 1 root root 0 Apr 14 13:22 a
-rw-r--r-- 1 root root 0 Apr 14 13:22 b

Looking into the macOS host's filesystem we see the same permissions for the two files

~/tmp/dockerchmod % ls -l macfs 
total 0
-r--r--r--  1 mgd  staff  0 Apr 14 15:22 a
-rw-r--r--  1 mgd  staff  0 Apr 14 15:22 b

In the container, we can change the permissions for file a:

root@5f0081ad212e:~/macfs# chmod u+w a
root@5f0081ad212e:~/macfs# ls -l a
-rw-r--r-- 1 root root 0 Apr 14 13:22 a

And we see it reflected in the macOS filesystem:

~/tmp/dockerchmod % ls -l macfs/a
-rw-r--r--  1 mgd  staff  0 Apr 14 15:22 macfs/a

Using VirtioFS sharing

If “VirtioFS” is selected as file sharing implementation, files that are readonly on the macOS host's filesystem cannot have their permissions manipulated in the container.

After removing the files from the previous experiment and switching to “VirtioFS” sharing, we launch a new container instance and create the same two files:

~/tmp/dockerchmod % docker run --rm -it -v$(pwd)/macfs:/root/macfs debian
root@2057dacfa2fa:/# cd ~/macfs/
root@2057dacfa2fa:~/macfs# (umask 222; touch a)
root@2057dacfa2fa:~/macfs# touch b
root@2057dacfa2fa:~/macfs# ls -l  
total 0
-r--r--r-- 1 root root 0 Apr 14 13:29 a
-rw-r--r-- 1 root root 0 Apr 14 13:29 b

The permissions look the same outside the container:

~/tmp/dockerchmod % ls -l macfs 
total 0
-r--r--r--  1 mgd  staff  0 Apr 14 15:29 a
-rw-r--r--  1 mgd  staff  0 Apr 14 15:29 b

However, attempting to modify permissions for file a inside the container fails:

root@2057dacfa2fa:~/macfs# chmod u+w a
chmod: changing permissions of 'a': Permission denied

Changing the permissions of file b is possible:

root@2057dacfa2fa:~/macfs# chmod u-w b
root@2057dacfa2fa:~/macfs# ls -l b
-r--r--r-- 1 root root 0 Apr 14 13:29 b

However, The permissions for b has not changed outside the container:

~/tmp/dockerchmod % ls -l macfs/b
-rw-r--r--@ 1 mgd  staff  0 Apr 14 15:29 macfs/b

Please note that file b now has extended attributes set so we dump them:

~/tmp/dockerchmod % xattr macfs/b
com.docker.grpcfuse.ownership

We can change b back inside the container:

root@2057dacfa2fa:~/macfs# chmod u+w b
root@2057dacfa2fa:~/macfs# ls -l b
-rw-r--r-- 1 root root 0 Apr 14 13:29 b

But if we remove w from file b in the host's filesystem:

~/tmp/dockerchmod % ls -l macfs 
total 0
-r--r--r--  1 mgd  staff  0 Apr 14 15:29 a
-r--r--r--@ 1 mgd  staff  0 Apr 14 15:29 b

Then, this is not reflected inside the container:

root@2057dacfa2fa:~/macfs# ls -l
total 0
-r--r--r-- 1 root root 0 Apr 14 13:29 a
-rw-r--r-- 1 root root 0 Apr 14 13:29 b

Now, neither a nor b can have their permissions changed inside the container:

root@2057dacfa2fa:~/macfs# chmod u+w a b
chmod: changing permissions of 'a': Permission denied
chmod: changing permissions of 'b': Permission denied
root@2057dacfa2fa:~/macfs# ls -l
total 0
-r--r--r-- 1 root root 0 Apr 14 13:29 a
-rw-r--r-- 1 root root 0 Apr 14 13:29 b
root@2057dacfa2fa:~/macfs# chmod u-w a b
chmod: changing permissions of 'a': Permission denied
chmod: changing permissions of 'b': Permission denied
root@2057dacfa2fa:~/macfs# ls -l
total 0
-r--r--r-- 1 root root 0 Apr 14 13:29 a
-rw-r--r-- 1 root root 0 Apr 14 13:29 b

However, we can still delete the files inside the container:

root@2057dacfa2fa:~/macfs# rm a b
root@2057dacfa2fa:~/macfs# ls -l
total 0

And, they are also deleted from the host filesystem:

~/tmp/dockerchmod % ls -l macfs 
total 0

[fredericdalleau]

@martingd Thanks for feedback. It's a file system limitation at the moment : if write permission is removed on a file, it can't be added back from the VM. Some containers in our test setup used to do that and other weird things. To mitigate this behavior, extended attributes are used to prevent removal of this permission on the host from containers.
Can you describe your use case a bit more precisely?

[martingd]

@fredericdalleau Thanks for the reply.

The concrete use case where I ran into this problem is:

• I start a container that maps a folder on the host.
• Inside the container, in the mapped folder, I unpack a tar archive.
• One of the files in the tar archive contains a file with -r--r--r-- permissions.

Now, this file cannot have its permissions changed.

Example:

~/tmp/dockerchmod% docker run --rm -it -v$(pwd)/macfs:/root/macfs debian
root@1d8a3135ef9d:/# cd   ~/macfs
root@1d8a3135ef9d:~/macfs# ls -l
total 4
-rw-r--r-- 1 root root 153 Apr 25 07:52 archive.tgz
root@1d8a3135ef9d:~/macfs# tar xzf archive.tgz 
root@1d8a3135ef9d:~/macfs# ls -l
total 4
drwxr-xr-x 4  503 staff 128 Apr 25 07:48 archive
-rw-r--r-- 1 root root  153 Apr 25 07:52 archive.tgz
root@1d8a3135ef9d:~/macfs# cd archive
root@1d8a3135ef9d:~/macfs/archive# ls -l
total 0
-r--r--r-- 1 root root 0 Apr 25 07:48 a
-rw-r--r-- 1 root root 0 Apr 25 07:48 b
root@1d8a3135ef9d:~/macfs/archive# chmod 644 a
chmod: changing permissions of 'a': Permission denied

Looking into the host's filesystem, we see the file a has been created readonly:

~/tmp/dockerchmod/macfs/archive 👉 ls -l
total 0
-r--r--r--  1 mgd  staff  0 Apr 25 09:48 a
-rw-r--r--  1 mgd  staff  0 Apr 25 09:48 b

Inside the container, I can still rm the file but not change its permissions:

root@1d8a3135ef9d:~/macfs/archive# rm a b
root@1d8a3135ef9d:~/macfs/archive# ls -l
total 0

[fredericdalleau]

Hi, to keep you posted, the issue is reproduced. It turns out tar uses a little bit different way to create a file on the host. There are multiple paths to fix it that are being discussed. As a workaround, you can extract your file in a volume and change the permissions there, and then move it on the share.

[martingd]

@fredericdalleau A similar (the same) workaround is to extract the files in /tmp (in the container's filesystem), fix the permissions and then move it onto the share.

Is a fix scheduled for this issue?

[martingd]

@fredericdalleau any update on this ticket?

[DougPlumley]

Hi, to keep you posted, the issue is reproduced. It turns out tar uses a little bit different way to create a file on the host. There are multiple paths to fix it that are being discussed. As a workaround, you can extract your file in a volume and change the permissions there, and then move it on the share.

Any updates on this? Looks like Docker is pushing for VirtioFS to be the default file sharing framework but as far as I know this issue has not been resolved.

Thanks!

[Elidrake]

Agreeing with Doug, this has not been resolved. I'm able to see this issue when running the following Pihole image:

version: "3"

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    # For DHCP it is recommended to remove these ports and instead add: network_mode: "host"
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp" # Only required if you are using Pi-hole as your DHCP server
      - "1953:80/tcp"
    environment:
      FTLCONF_CHECK_DISK: '0' # https://github.com/pi-hole/docker-pi-hole/issues/951
      TZ: 'America/Chicago'
      WEBPASSWORD: "*******"
    # Volumes store your data between container upgrades
    volumes:
      - './etc-pihole:/etc/pihole'
      - './etc-dnsmasq.d:/etc/dnsmasq.d'
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN # Recommended but not required (DHCP needs NET_ADMIN)
    restart: always

Mapped directories work fine under osxfs and gRPC FUSE, but the software simply errors out when attempting to access the same files using VirtioFS.

[Akazm]

I can reproduce this issue even without using VirtioFS.

[inventumamet]

Any updates on this issue? This is preventing me from using VirtioFS for some of the work I had to do where I need to use tar files. The script for extraction of the tar files are in another software package that I don't have access to modify the source. Hence, the suggested work around doesn't work for me. Instead, I need the docker to be able to extract the tar files with correct permissions to start with.

As long as the osxfs is going to be supported indefinitely into the future, then that is okay for me. Otherwise, please help fix this issue. If you are are out of resources, then please point me at where to go look for the fix to this, and I will submit a pull request.

[eduleon]

We also have issues with VirtioFS.

Not so sure if it's related with permissions or timestamps, but when we deploy EAR files, using a bind mount, the app server falls into an infinite deploy loop.

The rest of the file sharing implementations (gRPC FUSE and osxfs) don't cause the issue. Fortunately.

[cadenkriese]

Another reproducible scenario is using restic

I set up a test environment with the following structure:

.
├── backup
└── data
   └── data.txt

I then successfully run

~/tmp
❯ docker run -v ./data:/data:ro -v ./backup:/backup -e RESTIC_REPOSITORY='/backup' \
-e RESTIC_PASSWORD='test' \
restic/restic init
created restic repository fd324de92e at /backup

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

After which, the following command fails (worth noting it fails with or without the :ro specification)

~/tmp
❯ docker run -v ./data:/data:ro -v ./backup:/backup -e RESTIC_REPOSITORY='/backup' \
-e RESTIC_PASSWORD='test' \
restic/restic backup /data --verbose
open repository
lock repository
no parent snapshot found, will read all files
load index files
start scan on [/data]
start backup on [/data]
scan finished in 0.208s: 1 files, 15 B
error: read /data/data.txt: input/output error

Files:           0 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Data Blobs:      0 new
Tree Blobs:      0 new
Added to the repository: 0 B   (0 B   stored)

processed 0 files, 0 B in 0:00
snapshot c63d5cfd saved
Warning: at least one source file could not be read

and of course, all is fine when using gRPC FUSE.

[quacktacular]

Just a note observing the same issue (in my case impacting ability to share a dir used by gpg agent)

[adamnovak]

This seems to have the result that there are some tar archives that just can't be decompressed into a mounted directory at all.

For example, the LLVM release tarballs. When I try to do:

curl -LJ https://releases.llvm.org/6.0.1/cfe-6.0.1.src.tar.xz | tar -Jx

I get a bunch of errors like:

tar: cfe-6.0.1.src/test/Driver/Inputs/multilib_32bit_linux_tree/usr/i386-unknown-linux/bin/ld: Cannot open: Permission denied
tar: cfe-6.0.1.src/test/Driver/Inputs/multilib_32bit_linux_tree/usr/i386-unknown-linux/bin/as: Cannot open: Permission denied
tar: cfe-6.0.1.src/test/Driver/Inputs/multilib_64bit_linux_tree/usr/x86_64-unknown-linux/bin/ld: Cannot open: Permission denied
tar: cfe-6.0.1.src/test/Driver/Inputs/multilib_64bit_linux_tree/usr/x86_64-unknown-linux/bin/as: Cannot open: Permission denied

Are there particular options to tar that must be used to allow it to decompress all archives to host-mounted paths under Docker for Mac?

Using gRPC FUSE is not a usable workaround in our environment, because with that system sometimes a file can appear as zero-length when mounted into the container (see chanzuckerberg/miniwdl#461).