Cannot delete files/folders on Windows 11 when fsmonitor--daemon running.

[thargy]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-op

git version 2.32.0.windows.2
cpu: x86_64
built from commit: 3d45ac813c4adf97fe3733c1f763ab6617d5add5
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.22000.132]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VisualStudioCode
Custom Editor Path:
Default Branch Option: main
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Enabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Enabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Clean install of Windows 11

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

All the above

• What commands did you run to trigger this issue?

Tried to delete files/folders in Windows Explorer and/or from command line

• What did you expect to occur after running these commands?

Expected to be able to delete them

• What actually happened instead?

Was told couldn't delete as files/folders were in use by another process.

Workaround

Deleting the .git subfolder allowed deletion to occur.

[dscho]

FSMonitor does not access files or folders, as far as I understand, it only holds a handle on the top-level directory (and on the .git directory). @jeffhostetler please correct me if I am wrong.

If I am right, then the problem is more likely another process, maybe Defender, maybe an editor such as VS Code (which calls git status semi-frequently, which would temporarily lock the files and folders).

[thargy]

FSMonitor does not access files or folders, as far as I understand, it only holds a handle on the top-level directory (and on the .git directory). @jeffhostetler please correct me if I am wrong.

If I am right, then the problem is more likely another process, maybe Defender, maybe an editor such as VS Code (which calls git status semi-frequently, which would temporarily lock the files and folders).

I killed VSCode and Defender when trying to figure out what was holding locks. I only discovered it was fsmonitor when I opened up Resource Manager and searched for the handles.

[thargy]

OK, so some more diagnoses. Here's a fully reproducible set of steps that seems to work pretty reliably. I enter each command one at a time (rather than running a batch):

# Clone github repro (contains only README.md)
$ gh repo clone Code-Institute-Solutions/SampleREADME
Cloning into 'SampleREADME'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 4.87 KiB | 4.87 MiB/s, done.

# Empty README.md
$ > SampleREADME/README.md

# Check status
$ git -C SampleREADME/ status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

# Check log (checking status then log seems to reliably trigger locking)
$ git -C SampleREADME/ log
commit 982e3b0e0086ced89d9a643d9e4d9d0c6f938ec2 (HEAD -> master, origin/master, origin/HEAD)
Author: Matt Rudge <[email protected]>
Date:   Mon May 11 15:55:31 2020 +0000

    docs: Initial README.md commit

# Wait until git.exe grabs handles
# Grab below screenshot

# Try to delete directory
$ rm -rf SampleREADME/
rm: cannot remove 'SampleREADME/': Device or resource busy

# Sometimes the handles will terminate on there own, after seconds or minutes, otherwise the following works

# Remove .git directory
$ rm -rf SampleREADME/.git

# Can now delete
$ rm -rf SampleREADME/

Here's the screenshot referenced above:

As you can see, I never enter the directory directly in bash, or use VSCode. Also, you can see that code is not running from the arrow on the Processes view (sorted alphabetically). Most importantly git.exe holds all the handles, and mostly on .git\fsmonitor--daemon.ipc.

[jeffhostetler]

The fsmonitor-daemon process has a handle on the root of the working directory to receive notification events and it's CWD is set to that directory. The CWD is a problem and is probably the reason you can't delete the directory.

There is code in there to detect if the .git directory is deleted and auto shutdown. So there's a race there (since the recursive rmdir will delete the .git dir first and then the root directory) so sometimes the rmdir succeeds and sometimes not.

I'm have a todo item to have the daemon cd out of the root dir after it gets started up. that should fix the problem.

[thargy]

Thanks, @jeffhostetler, hopefully, my repro-steps will help you test the scenario to see if it resolves the issue. If you tag this issue when merged I'm happy to confirm it WFM too.

[mcardia]

I have the same problem that @thargy reported.
I found out that the git process keeps running and killing it by force releases the locked files.

git --version --build-options
git version 2.33.0.windows.2
cpu: x86_64
built from commit: 8735530946cced809cc6cc4c2ca3b078cdb3dfc8
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

Microsoft Windows [versão 10.0.22000.194]

[jeffhostetler]

I have a fix for this and it will be in the next release. In it, the daemon will CD out of the worktree root directory before starting its work loop.

For now, you can use git fsmonitor--daemon stop in the repo before trying to delete the worktree.

To address @thargy comments, this is the same problem. The daemon itself (as all Git commands do at startup) will CD into the root directory (using the value of the -C argument) -- and this is what is causing the lock on the root directory. I have code in there to detect when the .git directory is removed (which is easy since it is inside the watch-cone) and try to quickly shutdown (anticipating that the rest of the worktree is also about to disappear), but that is racy and sometimes we win and sometimes we lose. A proper fix will be in the next release.

Thanks
Jeff

[dscho]

A proper fix will be in the next release.

@jeffhostetler isn't this fix in the latest snapshot? If so, we should ask @thargy to test. A Git for Windows v2.33.1 is imminent, and it would be really good if @thargy could verify a fix for this problem before we release this version.

[thargy]

Errr... 😦 I reinstalled windows 11 since I last tested and then I've been mostly working in Ubuntu recently. Nonetheless, I'll try to get a test done. 👍🏻

[thargy]

OK, I have confirmed that my reproducable steps from above still work reliably on a completely new install. Indeed I reinstalled GitForWindows over the existing installation, just adding back in the fsdameon, and it went from not having the problem to having the problem again 👍

I then downloaded the latest snapshot as of 13 October 16:47:05 and installed it.

Running through the above steps again, I confirmed that the git.exe (specifically fsdaemon) was grabbing the filehandles, however, I could recursively delete the parent folder without issue.

I repeated the test several times without failure, whereas it would fail consistently before.

As such, I am reasonably confident the snapshot fixes the issue. 🎆

Hopefully, that is of service! I'm not sure what your policy on issues is so feel free to close when appropriate (i.e. do you close on merge or release?)

[dscho]

@thargy one last thing, could you upgrade to the just-released v2.33.1 and verify that the issue is still fixed? (I'll preemptively close this here ticket.)

[mcardia]

I've tested as well, and I confirm that the bug was fixed.

The git process with fsmonitor--daemon no longer locks files, and it ends itself when the repository is deleted.

Thank you.

[thargy]

@thargy one last thing, could you upgrade to the just-released v2.33.1 and verify that the issue is still fixed? (I'll preemptively close this here ticket.)

As with @mcardia, I have retested with the v2.33.1 release using the above reproducable steps and can confirm this issue is now resolved.

[dscho]

@mcardia @thargy thank you!

[jeffhostetler]

@mcardia @thargy Yes, thanks!!!