Unable to rename open file on Windows volume mount (grpc-fuse)

[awprice]

• I have tried with the latest version of my channel (Stable or Edge)
• I have uploaded Diagnostics
• Diagnostics ID: 0A16576C-6363-47F7-9CA3-2103004A0881/20200123110700

Expected behavior

When renaming an open file, I expect the renameat syscall to complete successfully.

Actual behavior

When renaming an open file, the renameat syscall fails with -1 EBUSY (Device or resource busy).

Information

This problem is easily reproducible. I've come up with the following Go code that can be used to replicate:

package main

import (
	"log"
	"os"
)

func main() {
	f, err := os.Create("./test")
	if err != nil {
		panic(err)
	}
	log.Println(f.Fd())
	if err  := os.Rename("./test", "./test2"); err != nil {
		panic(err)
	}
}

The code above creates and opens a file in the current directory - ./test with openat. It will then print the fd of the file, and then attempt to rename the file to ./test2 using renameat.

Under a normal Linux system, this completes without issue. When you run it on a filesystem that is a Windows volume mount, the code will panic due to an error experienced -

2020/01/23 11:14:39 3
panic: rename ./test ./test2: device or resource busy

goroutine 1 [running]:
main.main()
        /home/alexprice/go/src/github.com/awprice/docker-for-win-grpc-bug/main.go:15 +0x156

• Windows Version: Windows 10 Enterprise, Version 1903, Build 18362.592
• Docker Desktop Version: Stable, 2.2.0.0 (42247)
• Are you running inside a virtualized Windows e.g. on a cloud server or on a mac VM: No

Steps to reproduce the behavior

1. Ensure the desired drive is shared with Docker desktop, in my case it is C:\
2. Save the Go code above into a file in the current directory - main.go
3. Start a Go container, ensuring the volume mount is the same directory as the main.go file -
   docker run -it --rm -v "/host_mnt/c/Users/Alex Price/go/src/github.com/awprice/docker-for-win-grpc-bug:/shared" golang:1.13 bash
4. Change into the shared directory - cd /shared
5. Run the Go code - go run main.go

You should experience the issue:

root@748dbded7703:/shared# go run main.go
2020/01/23 11:23:17 3
panic: rename ./test ./test2: device or resource busy

goroutine 1 [running]:
main.main()
        /shared/main.go:15 +0x156
exit status 2

We can double check that /shared is on the grpcfuse filesystem:

root@748dbded7703:/# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          59G  2.1G   54G   4% /
<snipped>
grpcfuse        238G  216G   23G  91% /shared

If you move the go code to a non-shared directory and run it, it should complete without issue:

root@748dbded7703:/shared# mv main.go /
root@748dbded7703:/shared# cd /
root@748dbded7703:/# go run main.go
2020/01/23 11:24:23 3
root@748dbded7703:/# echo $?
0

I recommend also using strace on the Go code to further understand what is happening.

[StefanScherer]

Thanks for the easy repro code sample.
We can reproduce the behavior in DD 2.2.0.0 with grpcfuse. We also can reproduce the problem with Windows containers and also the Go example compiled for Windows and run locally on Windows. In both cases we see a SHARING_VIOLATION in Process Monitor running the example.

We know of problems in Go to golang/go#32088 and golang/go#34681 that doesn't allow to open files with FILE_SHARE_DELETE. That seems related to the problem and it looks like this won't be changed in Golang.
From what I can read in these tickets it is the current behavior. The chance that we can change this behavior is low.

It doesn't seem to happen with samba share in DD 2.1.0.5, it also works in DD 2.2.0.0 and the WSL2 backend (9P share).

As we're coming closer to the Windows 10 2004 release our focus is the WSL2 backend.

[awprice]

Thanks for the detailed response @StefanScherer!

For some context, I originally encountered the issue when trying to run RethinkDB - https://hub.docker.com/_/rethinkdb and share the databases' data directory with Windows so that it would persist.

It would try to rename a temporary file (which it also had open at the same time) - https://github.com/rethinkdb/rethinkdb/blob/98fd78409095a25b04dcaf0eadd6719f381a7d7e/src/serializer/log/log_serializer.cc#L88

[JoostvdB94]

Possible duplicate: #5665

[stephen-turner]

This is fixed in 2.2.0.4.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked