Convert paste line endings

[d-bingham]

Summary of the Pull Request

Implements line-ending conversion for pasting. Specifically by introducing a new option -- convertPasteLineEndings (optional, defaults to false) -- which, when set, causes pastes from the clipboard to have CRLF pairs converted to LFs.

See here: (Top copy/paste is prior behavior or convertPasteLineEndings set to false; bottom is convertPasteLineEndings set to true; this is Terminal running WSL bash):

References

PR Checklist

• Closes Pasted text includes CRLF pairs where unneeded #1091
• CLA signed. If not, go over here and sign the CLA
• Tests added/passed
• Requires documentation to be updated -- adds new option
• I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: Implement copy and paste for keyboard shortcuts #1072, Pasted text includes CRLF pairs where unneeded #1091

Detailed Description of the Pull Request / Additional comments

The new option, convertPasteLineEndings does what it says -- replacing Windows-space CRLF pairs with Unix-space LFs in text pasted to the console. This option applies to both the keyboard-shortcut paste and the right-click paste. Without this most multiline text pasted into Terminal will be "double-spaced" due to the Windows-style CRLF pairs. Furthermore, any multiline text copied from Terminal (in trim whitespace mode) generates CRLF pairs, which guarantees double-spacing when copying from a WSL Terminal session.

Changes:

• Added TermControl::_SendPastedTextToConnection which adds a pre-processing layer on top of _SendInputToConnection to allow line-ending conversion
• Various code in Profile/TerminalSettings to support the new option "convertPasteLineEndings" (optional, defaults to false)

Notes:

• Split from Implement copy and paste for keyboard shortcuts #1072

[d-bingham]

Windows -> Unix line-ending conversion is pretty much unavoidable for Terminal to interact happily with WSL. TextBuffer::GetTextForClipboard inserts CRLF pairs into copied text, and while this could be changed to (optionally) generate LFs instead of CRLFs, this doesn't seem to be sufficient since many/most Windows text editors (at a minimum Notepad, VS2019, and vscode) generate CRLF pairs.

I think fixing this at the paste level is the ideal location for the fix, as I'm not aware of another way for CRLFs to reasonably get into Unix-space terminal sessions that don't want them.

Fixing it as a profile-level option seems appropriate (as opposed to having separate key bindings for "raw paste" and "CRLF fixed paste"), as key bindings are app-wide and a profile-level setting allows different behavior for, say, bash vs. powershell.

[d-bingham]

This has overlap with #395 -- at the least, this fix and #395 seem to want to process the pasted text at the same place.

[ivan-section-io]

Generate LFs on paste instead of CRLFs? I think this is incorrect.

If this is a terminal and we're trying to emulate typing - I believe we're likely after CRs instead of CRLFs.

Some shells (like bash) look for LF endings, and then use stty icrnl setting to convert incoming interactive keyboard CRs to LFs. This way they can accept both CR or LF endings.

Unfortunately, if you start using a (typically interactive) program (e.g. nano editor) that doesn't use stty icrnl and thus can tell the difference between CRs and LFs - you'll find that it expects a CR for [Enter]/[Return] keys, and that sending an LF will cause "strange" behavior (e.g. turning LFs to spaces).

If you test pasting to both bash and nano in a default WSL Ubuntu - you'll find that CRLF to CR works in both, whereas CRLF to CR only works for bash.

(I've just re-tested all 3 CRLF/LF/CR line-ending pasting scenarios into WSL using 880272c, and CR line-endings was the only that worked as "expected".)

[d-bingham]

Updated to sending the correct character.

[d-bingham]

Follow-up: Is there any reason to have more-granular settings? That is, is there a point to having an option which specifies which line-ending to use, or is \x0d sufficient? Clearly my knowledge here is lacking, because I thought bash/vim's handling of \x0a was universal...

[ivan-section-io]

Follow-up: Is there any reason to have more-granular settings?
Good question.

Consumer scenarios I can think of:
Paste where line-endings should be converted to a Return/Enter key equivalent.
Paste to something expecting raw content... I have no idea what (practically) that might be.

Producer scenarios:
I'm likely to have clipboard content with Unix LF line endings, and would want that switched to CR terminal codes for pasting to interactive applications.

I'm just ignoring bracketed paste (http://www.xfree86.org/current/ctlseqs.html#Bracketed%20Paste%20Mode) at this point

[zadjii-msft]

Seems mostly reasonable to me.

This won't really work for profiles that have the setting set to false (like cmd.exe) and then run wsl though, correct? Hopefully in the future we can work on a solution to that scenario too.

[zadjii-msft]

As a tangential change, do we want the WSL profiles that are auto-generated to default to having this setting set to true?

[d-bingham]

I think that sounds like a good idea.

[mauve]

std::regex_replace allows replacing all matches in a single op:

if (_settings.ConvertPasteLineEndings()) {
  const static std::wregex rgx{ L"\n\r" };
  std::wstring stripped = std::regex_replace(wstr, rgx, L"\n");
  _SendInputToConnection(stripped);
}
...

[miniksa]

/azp run

[azure-pipelines]

Pull request contains merge conflicts.

[DHowett-MSFT]

I'm not sure this needs to be a setting. conpty should be turning <CR> into the right format on the other end. Maybe this is important for other types of connections?

Dustin's comment about conpty has me thinking maybe conpty is the place this should be fixed, not the terminal

[DHowett-MSFT]

It looks like this was outmoded by #2390. Thanks again for the original inspiration!