GH-72904: Add glob.translate() function

[barneygale]

Add glob.translate() function that converts a pathname with shell wildcards to a regular expression. The regular expression is used by pathlib to implement match() and glob().

This function differs from fnmatch.translate() in that wildcards do not match path separators by default, and that a * pattern segment matches precisely one path segment. When recursive is set to true, ** pattern segments match any number of path segments, and ** cannot appear outside its own segment.

In pathlib, this change speeds up directory walking (because _make_child_relpath() does less work), makes path objects smaller (they don't need a _lines slot), and removes the need for some gnarly code.

• Issue: '*' matches entire path in fnmatch #72904

---

📚 Documentation preview 📚: https://cpython-previews--106703.org.readthedocs.build/

[barneygale]

~20% globbing speedup:

$ ./python -m timeit -s 'from pathlib import Path; p = Path()' 'list(p.glob("**/*", follow_symlinks=False))'
2 loops, best of 5: 175 msec per loop  # before
2 loops, best of 5: 146 msec per loop  # after

[barneygale]

I've moved this to a new glob.translate() function.

It was easy enough to implement a recursive argument, so I did that and made its default False to match glob().

It's much harder to implement an include_hidden argument, so I've left that for now. I don't feel great about it, tbh.

[barneygale]

Right, after some futzing around I'm going to mark this PR as ready again.

In fnmatch.py, I've split the translate() method into _translate() and _join_translated_parts(). This minimises the diff, risk, and performance impact in that module.

(In #109879 I've re-implemented fnmatch.translate(), but that PR is an optional side-quest now)

In glob.py, I've spent some time making the implementation of translate(include_hidden=False) as clear and performant as I can. There's still a fair whack of regex involved but I think it's followable.

In pathlib.py, I've made pattern matching use str(path) directly, rather than a slight variant that represents empty paths as '' rather than '.'.

[barneygale]

Timings:

$ ./python -m timeit -n 5 -s 'from glob import glob' 'list(glob("**/*", recursive=True, include_hidden=True))'
5 loops, best of 5: 105 msec per loop  # for interest
$ ./python -m timeit -n 5 -s 'from pathlib import Path; p = Path()' 'list(p.glob("**/*", follow_symlinks=True))'
5 loops, best of 5: 86.6 msec per loop  # before
5 loops, best of 5: 72.7 msec per loop  # after

So Path.glob() is ~20% faster than before, and ~45% faster than glob.glob(), at least for this test case.

[AA-Turner]

A desk review:

A

[barneygale]

I've realised that the docs are a bit skew-whiff. Fix is in a separate PR: #110418

[encukou]

Is it correct to keep duplicated path separators?

>>> glob.translate('a//b')
'(?s:a//b)\\Z'

[barneygale]

Is it correct to keep duplicated path separators?

>>> glob.translate('a//b')
'(?s:a//b)\\Z'

glob() keeps them:

>>> os.makedirs('a/b')
>>> glob.glob('a//b')
['a//b']

So I reckon yes?

[barneygale]

Also, the number of additional slashes is meaningful in some cases, e.g. in Windows UNC paths or POSIX paths starting with two forward slashes. I don't think a pattern like /foo should match a path like //foo, and vice-versa.

[barneygale]

Hey @encukou, do you think I can merge this, or should I wait for a more complete review from someone?

[encukou]

Oh, I should have been more clear that I wouldn't get to a thorough review in any reasonable time.
You already have a few approvals, merge away!