$ is not line-ending aware

[sbromberger]

Summary

In vim, $ represents the end of the line regardless of how the line ending is encoded (specifically for this issue, CRLF vs LF). In helix, $ appears to map only to \n.

Having $ map to the end-of-line sequence (\r\n for CRLF encodings) would obviate the need to understand the line encoding of the file being edited, so that searches for things at the end of the line can be consistent.

Reproduction Steps

1. Create a file with CRLF line endings and a comma as the last character of a line.
2. Search for "comma followed by line ending" which is typically /,$ - it will not be found.
3. Search for /,\r$ - it will be found.

Repeating for a file with LF line endings, step 2 will succeed, and step 3 will fail.

Platform

Linux

Terminal Emulator

blink.sh / mosh

Helix Version

22.08.1-121-g8e363178

[kirawi]

rust-lang/regex#244

According to the upstream issue, it looks like using \r?$ is the best workaround.

[sbromberger]

But this may change:

In particular, I personally think the best path forward is to support both \r\n and \n as line endings, while ignoring the rest (\r on its own, as well as the rest of the line endings recommended by UTS#18).

[sbromberger]

Related question: is helix beholden to whatever the rust regex package decides? That is, helix could interpret $ as \r?$ internally, and that would "fix" helix.

[dead10ck]

I totally empathize with the issue here, and I absolutely agree \r\n should be seen as an eol. However, I shudder to start implementing subtle changes to the regex language in use, which amounts to starting a new dialect.

But also, that issue has been open since 2016, so who knows how long it will take to get fixed?

I'm really torn on this one.

[sbromberger]

I get it - but consider the argument that helix is using the regex library a bit differently than most developers use a regex library. That is, file-based regex parsing is different from stream- or data-based parsing in a few ways, line termination among them*.

Fundamentally they're the same thing, but an argument could be made that the implementations/usages diverge enough that it might not be completely out of line to make some changes that benefit Helix's file-based use of regex more than other uses.

Just food for thought. I'm torn too but this particular issue caused me a couple hours of grief. (It was only because I saw the CRLF indicator in the statusline that caused me to search for a solution. That, and a question posed on Matrix.)

*Edited to clarify: the main difference is that line endings are almost always significant in file-based regex parsing, since eol is typically the "record terminator", but are only sometimes significant in stream/data-based parsing.

[BurntSushi]

I'm expecting $ (and ^) to be capable of being CRLF aware once rust-lang/regex#656 lands. I don't have a specific timeline, but I'm hoping for the next few months.

If y'all wouldn't mind reviewing my summary here to make sure it looks right to you, that'd be great: rust-lang/regex#244 (comment)

Also note that ripgrep has its own work-around for this issue (which I hope to remove once this lands in the regex engine), and it essentially works by rewriting $ as (?:\r??$). See: https://docs.rs/grep-regex/latest/grep_regex/struct.RegexMatcherBuilder.html#method.crlf

[pascalkuthe]

This has been fixed upstream

[abdavis]

Yes, the issue has been fixed upstream, but reading the linked issue seems to say the fix is opt in. In order to match crln as end of line, you have to prepend a flag. In the current version, to find a comma before a line ending, the regex looks like this: (?R),$. This can be made default behavior in the source code like this: regex::RegexBuilder::new(r",$").crlf(true).build().

Yes, technically you can get the wanted behavior with that R flag, but I believe it makes sense to make this the default behavior. Could we reopen this issue? I'd be happy to take a stab at implementing this.