Remove unneeded VT-specific control character handling (#4289)

## Summary of the Pull Request

This PR removes all of the VT-specific functionality from the `WriteCharsLegacy` function that dealt with control characters, since those controls are now handled in the state machine when in VT mode. It also removes most of the control character handling from the `Terminal::_WriteBuffer` method for the same reason.

## References

This is a followup to PR #4171

## PR Checklist
* [x] Closes #3971
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/Terminal) and sign the CLA
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #780 (comment)

## Detailed Description of the Pull Request / Additional comments

There are four changes to the `WriteCharsLegacy` implementation:

1. The `TAB` character had special case handling in VT mode which is now no longer required. This fixes a bug in the Python REPL editor (when run from a cmd shell in Windows Terminal), which would prevent you tabbing past the end of the line. It also fixes #3971.

2. Following on from point 1, the `WC_NONDESTRUCTIVE_TAB` flag could also now be removed. It only ever applied in VT mode, in which case the `TAB` character isn't handled in `WriteCharsLegacy`, so there isn't a need for a non-destructive version.

3. There used to be special case handling for a `BS` character at the beginning of the line when in VT mode, and that is also no longer required. This fixes an edge-case bug which would prevent a glyph being output for code point 8 when `ENABLE_PROCESSED_OUTPUT` was disabled. 

4. There was quite a lot of special case handling for control characters in the "end-of-line wrap" implementation, which is no longer required. This fixes a bug which would prevent "low ASCII" characters from wrapping when output at the end of a line.

Then in the `Terminal::_WriteBuffer` implementation, I've simply removed all control character handling, except for `LF`. The Terminal is always in VT mode, so the control characters are always handled by the state machine. The exception for the `LF` character is simply because it doesn't have a proper implementation yet, so it still passes the character through to `_WriteBuffer`. That will get cleaned up eventually, but I thought that could wait for a later PR.

Finally, with the removal of the VT mode handling in `WriteCharsLegacy`, there was no longer a need for the `SCREEN_INFORMATION::InVTMode` method to be publicly accessible. That has now been made private.

## Validation Steps Performed

I've only tested manually, making sure the conhost and Windows Terminal still basically work, and confirming that the above-mentioned bugs are fixed by these changes.

src/cascadia/TerminalCore/ITerminalApi.hpp

@@ -28,6 +28,7 @@ namespace Microsoft::Terminal::Core
 
         virtual bool SetCursorPosition(short x, short y) noexcept = 0;
         virtual COORD GetCursorPosition() noexcept = 0;
+        virtual bool CursorLineFeed(const bool withReturn) noexcept = 0;
 
         virtual bool DeleteCharacter(const size_t count) noexcept = 0;
         virtual bool InsertCharacter(const size_t count) noexcept = 0;

src/cascadia/TerminalCore/Terminal.cpp

@@ -414,7 +414,6 @@ Viewport Terminal::_GetVisibleViewport() const noexcept
 void Terminal::_WriteBuffer(const std::wstring_view& stringView)
 {
     auto& cursor = _buffer->GetCursor();
-    const Viewport bufferSize = _buffer->GetSize();
 
     // Defer the cursor drawing while we are iterating the string, for a better performance.
     // We can not waste time displaying a cursor event when we know more text is coming right behind it.
@@ -425,104 +424,85 @@ void Terminal::_WriteBuffer(const std::wstring_view& stringView)
         const auto wch = stringView.at(i);
         const COORD cursorPosBefore = cursor.GetPosition();
         COORD proposedCursorPosition = cursorPosBefore;
-        bool notifyScroll = false;
 
-        if (wch == UNICODE_LINEFEED)
-        {
-            proposedCursorPosition.Y++;
-        }
-        else if (wch == UNICODE_CARRIAGERETURN)
+        // TODO: MSFT 21006766
+        // This is not great but I need it demoable. Fix by making a buffer stream writer.
+        //
+        // If wch is a surrogate character we need to read 2 code units
+        // from the stringView to form a single code point.
+        const auto isSurrogate = wch >= 0xD800 && wch <= 0xDFFF;
+        const auto view = stringView.substr(i, isSurrogate ? 2 : 1);
+        const OutputCellIterator it{ view, _buffer->GetCurrentAttributes() };
+        const auto end = _buffer->Write(it);
+        const auto cellDistance = end.GetCellDistance(it);
+        const auto inputDistance = end.GetInputDistance(it);
+
+        if (inputDistance > 0)
         {
-            proposedCursorPosition.X = 0;
-        }
-        else if (wch == UNICODE_BACKSPACE)
-        {
-            if (cursorPosBefore.X == 0)
-            {
-                proposedCursorPosition.X = bufferSize.Width() - 1;
-                proposedCursorPosition.Y--;
-            }
-            else
-            {
-                proposedCursorPosition.X--;
-            }
-        }
-        else if (wch == UNICODE_BEL)
-        {
-            // TODO: GitHub #1883
-            // For now its empty just so we don't try to write the BEL character
+            // If "wch" was a surrogate character, we just consumed 2 code units above.
+            // -> Increment "i" by 1 in that case and thus by 2 in total in this iteration.
+            proposedCursorPosition.X += gsl::narrow<SHORT>(cellDistance);
+            i += inputDistance - 1;
         }
         else
         {
-            // TODO: MSFT 21006766
-            // This is not great but I need it demoable. Fix by making a buffer stream writer.
-            //
-            // If wch is a surrogate character we need to read 2 code units
-            // from the stringView to form a single code point.
-            const auto isSurrogate = wch >= 0xD800 && wch <= 0xDFFF;
-            const auto view = stringView.substr(i, isSurrogate ? 2 : 1);
-            const OutputCellIterator it{ view, _buffer->GetCurrentAttributes() };
-            const auto end = _buffer->Write(it);
-            const auto cellDistance = end.GetCellDistance(it);
-            const auto inputDistance = end.GetInputDistance(it);
-
-            if (inputDistance > 0)
-            {
-                // If "wch" was a surrogate character, we just consumed 2 code units above.
-                // -> Increment "i" by 1 in that case and thus by 2 in total in this iteration.
-                proposedCursorPosition.X += gsl::narrow<SHORT>(cellDistance);
-                i += inputDistance - 1;
-            }
-            else
-            {
-                // If _WriteBuffer() is called with a consecutive string longer than the viewport/buffer width
-                // the call to _buffer->Write() will refuse to write anything on the current line.
-                // GetInputDistance() thus returns 0, which would in turn cause i to be
-                // decremented by 1 below and force the outer loop to loop forever.
-                // This if() basically behaves as if "\r\n" had been encountered above and retries the write.
-                // With well behaving shells during normal operation this safeguard should normally not be encountered.
-                proposedCursorPosition.X = 0;
-                proposedCursorPosition.Y++;
-            }
+            // If _WriteBuffer() is called with a consecutive string longer than the viewport/buffer width
+            // the call to _buffer->Write() will refuse to write anything on the current line.
+            // GetInputDistance() thus returns 0, which would in turn cause i to be
+            // decremented by 1 below and force the outer loop to loop forever.
+            // This if() basically behaves as if "\r\n" had been encountered above and retries the write.
+            // With well behaving shells during normal operation this safeguard should normally not be encountered.
+            proposedCursorPosition.X = 0;
+            proposedCursorPosition.Y++;
         }
 
-        // If we're about to scroll past the bottom of the buffer, instead cycle the buffer.
-        const auto newRows = proposedCursorPosition.Y - bufferSize.Height() + 1;
-        if (newRows > 0)
-        {
-            for (auto dy = 0; dy < newRows; dy++)
-            {
-                _buffer->IncrementCircularBuffer();
-                proposedCursorPosition.Y--;
-            }
-            notifyScroll = true;
-        }
+        _AdjustCursorPosition(proposedCursorPosition);
+    }
 
-        // This section is essentially equivalent to `AdjustCursorPosition`
-        // Update Cursor Position
-        cursor.SetPosition(proposedCursorPosition);
+    cursor.EndDeferDrawing();
+}
 
-        const COORD cursorPosAfter = cursor.GetPosition();
+void Terminal::_AdjustCursorPosition(const COORD proposedPosition)
+{
+#pragma warning(suppress : 26496) // cpp core checks wants this const but it's modified below.
+    auto proposedCursorPosition = proposedPosition;
+    auto& cursor = _buffer->GetCursor();
+    const Viewport bufferSize = _buffer->GetSize();
+    bool notifyScroll = false;
 
-        // Move the viewport down if the cursor moved below the viewport.
-        if (cursorPosAfter.Y > _mutableViewport.BottomInclusive())
+    // If we're about to scroll past the bottom of the buffer, instead cycle the buffer.
+    const auto newRows = proposedCursorPosition.Y - bufferSize.Height() + 1;
+    if (newRows > 0)
+    {
+        for (auto dy = 0; dy < newRows; dy++)
         {
-            const auto newViewTop = std::max(0, cursorPosAfter.Y - (_mutableViewport.Height() - 1));
-            if (newViewTop != _mutableViewport.Top())
-            {
-                _mutableViewport = Viewport::FromDimensions({ 0, gsl::narrow<short>(newViewTop) }, _mutableViewport.Dimensions());
-                notifyScroll = true;
-            }
+            _buffer->IncrementCircularBuffer();
+            proposedCursorPosition.Y--;
         }
+        notifyScroll = true;
+    }
+
+    // Update Cursor Position
+    cursor.SetPosition(proposedCursorPosition);
 
-        if (notifyScroll)
+    const COORD cursorPosAfter = cursor.GetPosition();
+
+    // Move the viewport down if the cursor moved below the viewport.
+    if (cursorPosAfter.Y > _mutableViewport.BottomInclusive())
+    {
+        const auto newViewTop = std::max(0, cursorPosAfter.Y - (_mutableViewport.Height() - 1));
+        if (newViewTop != _mutableViewport.Top())
         {
-            _buffer->GetRenderTarget().TriggerRedrawAll();
-            _NotifyScrollEvent();
+            _mutableViewport = Viewport::FromDimensions({ 0, gsl::narrow<short>(newViewTop) }, _mutableViewport.Dimensions());
+            notifyScroll = true;
         }
     }
 
-    cursor.EndDeferDrawing();
+    if (notifyScroll)
+    {
+        _buffer->GetRenderTarget().TriggerRedrawAll();
+        _NotifyScrollEvent();
+    }
 }
 
 void Terminal::UserScrollViewport(const int viewTop)

src/cascadia/TerminalCore/Terminal.hpp

@@ -84,6 +84,7 @@ class Microsoft::Terminal::Core::Terminal final :
     bool ReverseText(bool reversed) noexcept override;
     bool SetCursorPosition(short x, short y) noexcept override;
     COORD GetCursorPosition() noexcept override;
+    bool CursorLineFeed(const bool withReturn) noexcept override;
     bool DeleteCharacter(const size_t count) noexcept override;
     bool InsertCharacter(const size_t count) noexcept override;
     bool EraseCharacters(const size_t numChars) noexcept override;
@@ -239,6 +240,8 @@ class Microsoft::Terminal::Core::Terminal final :
 
     void _WriteBuffer(const std::wstring_view& stringView);
 
+    void _AdjustCursorPosition(const COORD proposedPosition);
+
     void _NotifyScrollEvent() noexcept;
 
 #pragma region TextSelection

src/cascadia/TerminalCore/TerminalApi.cpp

@@ -132,6 +132,27 @@ COORD Terminal::GetCursorPosition() noexcept
     return newPos;
 }
 
+// Method Description:
+// - Moves the cursor down one line, and possibly also to the leftmost column.
+// Arguments:
+// - withReturn, set to true if a carriage return should be performed as well.
+// Return value:
+// - true if succeeded, false otherwise
+bool Terminal::CursorLineFeed(const bool withReturn) noexcept
+try
+{
+    auto cursorPos = _buffer->GetCursor().GetPosition();
+    cursorPos.Y++;
+    if (withReturn)
+    {
+        cursorPos.X = 0;
+    }
+    _AdjustCursorPosition(cursorPos);
+
+    return true;
+}
+CATCH_LOG_RETURN_FALSE()
+
 // Method Description:
 // - deletes count characters starting from the cursor's current position
 // - it moves over the remaining text to 'replace' the deleted text

src/cascadia/TerminalCore/TerminalDispatch.cpp

@@ -83,12 +83,9 @@ try
         // There is currently no need for mode-specific line feeds in the Terminal,
         // so for now we just treat them as a line feed without carriage return.
     case DispatchTypes::LineFeedType::WithoutReturn:
-        Execute(L'\n');
-        return true;
+        return _terminalApi.CursorLineFeed(false);
     case DispatchTypes::LineFeedType::WithReturn:
-        Execute(L'\r');
-        Execute(L'\n');
-        return true;
+        return _terminalApi.CursorLineFeed(true);
     default:
         return false;
     }