Regex excludes cause existing patterns to behave differently

[ckamm]

From @sagamusix #5017

for example my pattern */.svn/pristine/ no longer seems to work

Summary from investigation:

The client used fnmatch with FNM_PATHNAME on unix and PathMatchSpec on Windows. These had different behavior when expanding * patterns: fnmatch never expands it to /, while that is allowed on windows. That means */foo/bar would match a/b/foo/bar on windows, but not on other oses.

The new regex exclude matching code implements the fnmatch behavior on all platforms thereby changing the meaning of existing patterns on windows.

[ckamm]

@sagamusix So it's now syncing some folder foo/.svn/pristine and its contents?

[sagamusix]

Exactly, and previously these files were ignored as intended.

[ckamm]

@sagamusix It works for me. The pattern gets translated to ^[^/]*/\.svn/pristine)(?:$|/). Can you let me know what path exactly was excluded before and isn't anymore?

Idea I have: You're on windows, and we're seeing a previous inconsistency between windows and linux. On linux we used FNM_PATHNAME so a * never matched a /. Maybe on windows a * (incorrectly!) did match a slash?

Since we no longer rely on fnmatch and PathMatchSpec the behavior will now be consistent - and I used the fnmatch behavior as the baseline.

[sagamusix]

Two paths were matched before I realized and aborted the sync; one was Uni\powerwall-trunk\.svn\pristine and the other one was similar in structure. It seems like if I move the folder one level up, it works as intended, which coincides with your idea, I think.

[sagamusix]

I have changed the pattern to just .svn/pristine/ (including all other patterns in my list from other issue that were structured similarly) and it seems like the initial discovery is much, much slower now - it's now running for almost 2 hours for a ~35 GB folder that previously took a few minutes. In fact, it's still discovering folders so I cannot see yet if it helps fixing the problem. I can still occasionally see folders that are not supposed to be synced in the progress indication, though, so I'm not sure if this helped at all.

[ckamm]

@sagamusix changing too .svn/pristine/ still wouldn't trigger the right behavior. See #5275 for discussion of the unclear behavior.

[sagamusix]

So do I understand correctly that what I wanted previously only worked by chance?

[ckamm]

Yes, unfortunately. But I don't think it's acceptable to break patterns windows users had added. I think we'll need to put in extra work to make sure that migrates nicely.

And that performance issue you're seeing also definitely needs to be investigated.

[ckamm]

I'm looking at allowing * to match .* in the regex: it complicates the bname-as-activation optimization though since foo/bar*daa could very well be activated by a bname xyzdaa. It may be ok to be just mostly correct here.

[ckamm]

@sagamusix I've adjusted the regex matching on windows to be backwards compatible. Could you check out the new 2.5 nightly when it becomes available?

[sagamusix]

Judging from the timestamps, I guess the current nightly build doesn't have the fix yet, right?

[ckamm]

@sagamusix Available now: http://download.owncloud.com/desktop/daily/ownCloud-2.5.0.8967-nightly20180110-setup.exe

[sagamusix]

The patterns seem to work as intended again, although initial discovery still seems to have slowed down considerably. The discovery took almost exactly 5 hours, which used to be a matter of minutes. To make things worse, it ended with an "operation cancelled" error from the server. :)

[ckamm]

@sagamusix Ouch, that's horrible. Let's move back to the optimization task #5017. Thanks for testing.