"File exists" error on Windows

[kit-ty-kate]

Happened twice today in two different PRs

#=== ERROR while fetching sources for dune.3.6.1 and dune-configurator.3.6.1 ==#
OpamSolution.Fetch_fail("Failed to copy source of:\n  - dune.3.6.1: \"D:\\\\cygwin\\\\bin\\\\cp.exe -PRp C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\Temp\\\\opam-6696-56db09 D:\\\\a\\\\opam\\\\opam\\\\.opam\\\\default\\\\.opam-switch\\\\sources\\\\dune.3.6.1\" exited with code 1 \"/usr/bin/cp: cannot create regular file 'D:\\a\\opam\\opam\\.opam\\default\\.opam-switch\\sources\\dune.3.6.1/dune': File exists\"\n  - dune-configurator.3.6.1: \"D:\\\\cygwin\\\\bin\\\\cp.exe -PRp C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\Temp\\\\opam-6696-56db09 D:\\\\a\\\\opam\\\\opam\\\\.opam\\\\default\\\\.opam-switch\\\\sources\\\\dune-configurator.3.6.1\" exited with code 1 \"/usr/bin/cp: cannot create regular file 'D:\\a\\opam\\opam\\.opam\\default\\.opam-switch\\sources\\dune-configurator.3.6.1/dune': File exists\"\n")

Full logs:

• https://github.com/ocaml/opam/actions/runs/4166438520/jobs/7210802559
• https://github.com/ocaml/opam/actions/runs/4164409618/jobs/7206022465
• https://github.com/ocaml/opam/actions/runs/4173989001/jobs/7227012036

[dra27]

Expanding some of the escaping in the error:

#=== ERROR while fetching sources for dune.3.6.1 and dune-configurator.3.6.1 ==#
Failed to copy source of:
  - dune.3.6.1:
      "D:\cygwin\bin\cp.exe -PRp C:\Users\runneradmin\AppData\Local\Temp\opam-6696-56db09 D:\a\opam\opam\.opam\default\.opam-switch\sources\dune.3.6.1"
        exited with code 1 "/usr/bin/cp: cannot create regular file 'D:\\a\\opam\\opam\\.opam\\default\\.opam-switch\\sources\\dune.3.6.1/dune': File exists"
  - dune-configurator.3.6.1:
      "D:\cygwin\bin\cp.exe -PRp C:\Users\runneradmin\AppData\Local\Temp\opam-6696-56db09 D:\a\opam\opam\.opam\default\.opam-switch\sources\dune-configurator.3.6.1"
        exited with code 1 "/usr/bin/cp: cannot create regular file 'D:\\a\\opam\\opam\\.opam\\default\\.opam-switch\\sources\\dune-configurator.3.6.1/dune': File exists"

[dra27]

(just noting that the issue isn't necessarily repeating)

[jonahbeckford]

I've seen this problem before. I thought it was limited to MSYS2 not Cygwin because it is related to symlink copying. Let me see if I can scrounge for references:

• Fix: distinguish MSYS2 from Cygwin #5404
• -> https://github.com/ocaml/dune/blob/fa3aa2891ac74828a81f73894fe247f69c530c81/ci/build-test.sh#L82-L94
• -> https://github.com/ocaml/opam/blob/master/src/core/opamSystem.ml#L636-L694

The last reference gives a detailed description of what is happening and why it is indeterminate.

[db4]

In the case of Cygwin, I think I can explain what's going on. The problematic packages contain both dune and dune.exe files in the same directory, but Cygwin treats dune.exe as dune in some situations:

Cygwin always handled the .exe suffix transparently in terms of stat(2) calls, but Cygwin 1.7 also handles them transparently in terms of open(2) and any other call. Therefore, if a file foo.exe exists, and an application calls stat("foo"), it will get told that, yes, "foo" exists. That's a basic component of being able to call foo.exe from bash by just typing foo. POSIX systems just don't have the .exe suffix for executables.

(found in https://stackoverflow.com/questions/58048185/cygwin-cp-cant-copy-file-abc-because-destination-dir-contains-file-abc-exe)

That also explains why it works sometimes. If you copy a file with no extension first, you are lucky:

cp src/dune dst/dune
cp src/dune.exe dst/dune.exe
(OK)

But if you do the opposite, you are in trouble:

cp src/dune.exe dst/dune.exe
cp src/dune dst/dune
/usr/bin/cp: cannot create regular file 'dst/dune': File exists

This issue has effectively ruined my Windows build after upgrading some packages. How to fix it? I don't know. Never use dune and dune.exe together`?