Searching with inline regex backreference only searches open files

[bennor]

As far as I can tell, finding in files with a regular expression including a backreference (e.g. \1) will only return results from files that are open. This may be by design or due to some limitation I'm not aware of, but I couldn't find anything in the docs.

• VSCode Version: 1.12.2
• OS Version: Windows 10 (Version 1703 - Build 10.0.15063)

Steps to Reproduce:

1. Create a text file with the following text:

   ABCA
   ABC
   

2. Close the file.
3. Run a regex search across files for (A).*?\1. There are no results.
4. Open the file created in step 1.
5. Rerun the search. It matches the first line of the file.

[bennor]

Seems to work if I turn set search.useRipgrep to false, so maybe it's a limitation of Ripgrep?

[roblourens]

That's right, some regex features like backreferences and lookaround are not supported by ripgrep.

[roblourens]

I've seen several issues related to this, so maybe I should add a blurb in the docs?

[bennor]

Thanks. Is there any potential problem with disabling Ripgrep (other than performance)?

[bennor]

The only real problem I can see when Ripgrep is active is that it does find matches in open files (I'm not entirely sure how that works around the limitation of Ripgrep though), which can lead you to the mistaken assumption that you've found everything you were looking for.

[roblourens]

No, if you really need those features, you can disable ripgrep and just get a perf hit.

Search in open files is done by the editor, so the JS features work there. And ripgrep doesn't complain when we use an unsupported feature so it's hard to know that there may be other matches. Trying to detect unsupported patterns in the input regex makes me nervous but it may be worth it to avoid confusion.

The only reason I can think of that using ripgrep to search open files would be difficult is that it's really easy for the editor to do a search in a large file with unsaved changes, but slower to pipe all that text through ripgrep.

[roblourens]

Actually we do get an error when you try to use lookaround, which is good. Really it should raise an error for backreferences, but this issue: BurntSushi/ripgrep#268 - \1 is an octal escape sequence.

I think it would actually be fair for us to say, if the query contains \<n> and ripgrep returns no results, show an error that backreferences aren't supported. It's less likely that someone would use \1 or \2 or \3 in a search as an octal literal since you won't match a typical printable character until \11.

[BurntSushi]

@roblourens I think ripgrep should fix this. It won't happen right away so you may still want to create a work around (i.e., just outright reject octal escapes). More details: BurntSushi/ripgrep#268 (comment)

[roblourens]

Verifier - check that there is a warning printed when using a backreference like \1, and not for any other pattern, like a literal \ followed by a number.