End of line cursor position differences when app is run in Windows Terminal

[tlmii]

This isn't so much a bug report as I'm hoping for clarification on some behavior we're seeing in Windows Terminal related to a bug over at HttpRepl. Rich Turner suggested I file here to get more details.

I'm trying to understand the different behavior I'm seeing around the cursor position when our app (a dotnet global tool, though it doesn't appear specific to that type of app) is run from a normal cmd.exe prompt vs when it is run from within Windows Terminal, in a profile that launches cmd.exe. Specifically, this is about how the cursor position is updated when it is at the end of a line.

When our app does a Console.Write(char), in most cases we see the Console.CursorLeft (checked before and after the Write call) increase to the next slot after the character that was just written. Where we see the difference is when we Console.WriteLine(char) to the last slot on a line.

Running our app in vanilla cmd.exe, the character is written and the cursor visibly moves down to the beginning of the next line. Checking Console.CursorLeft and Console.CursorTop, we can see that the former reset to 0 and the latter increased by 1.

Running our app in cmd.exe launched in a Windows Terminal profile, the character is written, but the cursor disappears. Checking Console.CursorLeft and Console.CursorTop, we can see that neither of them changed - they now refer to the slot to which the character was just written.

Steps to reproduce

I set up a fairly minimal repro that demonstrates the difference. Run the app, start typing characters, and watch the upper-left of the console where we are printing the current Console.CursorLeft/Console.CursorTop position. Try it in both vanilla cmd.exe and cmd.exe from within Windows Terminal. Note the different behavior when you get to the last character on the line (and subsequent characters, actually, but I assume that's a derivative problem).

Note that the behavior is also different (in the same way) between a vanilla powershell prompt and a powershell prompt from within Windows Terminal.

Environment

It doesn't seem tied to a particular version of Windows or Terminal, but I've confirmed it on:

Windows build number: 10.0.20257.0
Windows Terminal version (if applicable): 1.5.3142.0

Expected behavior

I'm not asserting that this is unexpected behavior, necessarily. Instead I'm looking for clarity - maybe a little bit of "why" but also if it is possible for us to detect the different behavior so we can account for it.

Actual behavior

The behavior of the cursor position within a console app differs when it is run in Windows Terminal vs not.

[DHowett]

So, this is likely an effect of the console output modes ENABLE_WRAP_AT_EOL_OUTPUT and DISABLE_NEWLINE_AUTO_RETURN. They dictate whether the cursor wraps to the next line or "hangs" off the end of the line (technically) and report coordinates that look like they're at the end of the line.

Generally, we recommend that applications enforce the modes that work best for them 😄 since there's no guarantee that the shell that spawns them is going to set the same modes forever.

[DHowett]

If you interrogate the mode with GetConsoleMode on the output handle, are you getting different values when run from CMD in conhost (the traditional console window) versus Terminal?

[tlmii]

@DHowett Thanks for the quick reply!

It does look like there is a difference. Specifically, the DISABLE_NEWLINE_AUTO_RETURN was set for Windows Terminal, but not on conhost. Also ENABLE_LVB_GRID_WORLDWIDE was set for conhost but not for Windows Terminal, though that seems less relevant here.

However, it doesn't appear that changing that makes any difference in the behavior. I can modify it with SetConsoleMode, confirm it is set, and the subsequent behavior is still the same, both in HttpRepl and in the repro app I posted.

I'm not positive I did it correctly - used GetStdHandle(STD_INPUT_HANDLE) for the console window handle for the GetConsoleMode/SetConsoleMode calls and I'm not sure that's right. I thought this seemed like the exact thing I was looking for, so I was surprised it didn't make a difference.

Any thoughts?

[DHowett]

Hmmmm.. If you try it against the STD_OUTPUT_HANDLE instead, do you get a different result? The console mode is unfortunately split across those two handles (with the acceptable enum values coming from those two tables on the docs page.)

[tlmii]

Ah, darnit. I even looked at the two different lists and realized I needed to be using OUTPUT but didn't notice I had only tried STD_INPUT_HANDLE.

OK, so using STD_OUTPUT_HANDLE, I find that the results are all the same between the two ways of running it (conhost/terminal). ENABLE_WRAP_AT_EOL_OUTPUT is set on both and DISABLE_NEWLINE_AUTO_RETURN is not set on either.

But the one thing I noticed is that the difference appears to be the ENABLE_VIRTUAL_TERMINAL_PROCESSING setting. That's different between the working and non-working scenarios. And it turns out if I remove that flag, then things work as expected in Windows Terminal.

The descriptions for those two original flags describe what is happening very clearly. In HttpRepl, the combination of both being set would be exactly what we see - because it does move to the next line eventually on the subsequent character.

So its weird that the ENABLE_VIRTUAL_TERMINAL_PROCESSING flag is what makes a difference. Any idea why that would be? I recognize this might be straying from Windows Terminal-specific info, so feel free to redirect me elsewhere if needed.

[DHowett]

Oh, oh, okay. This makes sense.

First, though:

straying from Windows Terminal-specific info

I've got really good news: this repo is also home to conhost (the window CMD lives in!) (which also hosts all other Win32 console applications)

So, ENABLE_VIRTUAL_TERMINAL_PROCESSING is a fun one. There's a few key differences between traditional terminal emulators and the Windows Console. One of them is support for control sequences (cursor movement, color) in-band with text. Another one is cursor position handling. VT processing enables, implicitly, the aforementioned carriage handling behavior because that's what terminal emulators typically do.

We made a change back in #2824 to fix #1965. The salient comments, if you want to follow along, are #1965 (comment) and #1965 (comment). In short, we made Terminal (and any application like Terminal (VSCode, ConEmu in PTY mode, WSL in interop mode ...)) automatically enable ENABLE_VIRTUAL_TERMINAL_PROCESSING because they'll want their applications to act more like they would under terminals. There was a huge downside to us not enabling VT processing in those applications: sometimes they would emit text that tricked connected terminals into thinking that they were speaking in control codes when they actually weren't. We were aghast when we figured that one out.

Anyway, this is roughly somewhere between "by design" and "something the application can explicitly turn off", but I do accept that a small set of applications will find the observable oddity now and again. 😁

I'm going to mark this one as question/answered/closed, but don't let that deter you from asking further questions. I love talking about this stuff!

[tlmii]

Yes, this is all starting to make sense now. The timing of it all too - when this chunk of our code was written, that issue you linked to, and the PR didn't even exist yet. So... assumptions were made :) Thanks for all the detail!

but don't let that deter you from asking further questions. I love talking about this stuff!

Well, since you asked!

The question I posed in the original post was for one of two issues we ran into that impacts both conhost and wt (in slightly different ways). But since you offered, I could probably use a bit more clarification to make sure I understand it so we can properly work around it.

Running our app in WT, we see that the user is typing on the last line of the window (say, line 30), and the cursor reaches the end (say, position 120), the cursor position wraps back around to slot 0 (not on the next character, but on the following one, because of the above-discussed issues). But the line number doesn't change - its still on line 30. Checking Console.BufferHeight while running in WT shows the buffer height is the same as the number of lines on the screen. So that sort of makes sense - everything gets shifted up one since you're at the end.

We initially thought this problem didn't exist in plain cmd.exe. But it turned out that the difference was that the Console.BufferHeight was returning ~9000. If we filled that whole buffer, then ran our app, we still hit the same problem. So its not WT-specific.

It's clearly a bug on our side in how we handle that (the assumption that the row number would keep increasing in perpetuity) and I'm working on changing that code to handle it more appropriately. But I figured I could clarify if my understanding was right to make sure our fix is right.

I'm assuming that this difference in behavior is related to the "virtual" terminal idea. The "buffer" is only as big as it needs to be to render to the screen, even though the actually history is longer than that. So plain cmd.exe/conhost without VT processing enabled has a large buffer tied to the actual history, and then when it is in WT, with VT processing enabled, it is virtualized. Is that all there is to it? Or is there something more going on?

[DHowett]

Ah yeah, so that one's gonna be a big gotcha no matter where you're running.

The Terminal and the Console are actually the same beast, through-and-through[1]. The interface presented to the application--the console API--is actually serviced by an embedded instance of conhost (which when compiled out of this repository is called OpenConsole). So, behind every pane in Terminal there's a traditional console host. It's really cool!

That console host is run in a specific mode that (sometimes) catches applications out. I'll get to that in a sec.

That console host also handles translation from Win32 Console APIs to the standard control sequences that normal terminals everywhere except Windows understand. Terminal's built on top of that support: it doesn't need to know anything about the Win32 Console APIs, it just needs to know about VT. Together, we call these modes "ConPTY".

That mode that catches applications out, though. This is where it gets "fun".

We made a critical mistake approximately forty years ago (:grin:) and made the entire buffer (120x9001 or whatever the user requests) available to a console application. There's the viewport (120x30) and the buffer (120x9001), both of which the application can control. It can draw outside of its bounds, and it can know exactly where in the buffer it's printing. Pretty cool!

Applications that have some amount of buffer awareness will eventually reach a point where they're on line 9001 and oops the line under 9001 is also line 9001 (because the old line 0 got destroyed and shifted off the top of the buffer.) This bit is the crux of your issue 😄. We call it "circling."

Terminals have never supported that viewport/buffer split. They've only ever offered viewport control, and the buffer was an artifact owned by the terminal itself (because, like, it might have been a line printer or a serial device or a modem . . . instead of a terminal emulator application). That puts the terminal in charge of scrollback, and scrollback is stored in a place where the client application can't see (or change) it.

To account for this difference, ConPTY mode restricts the buffer size to match the viewport size. This restriction applies regardless of ENABLE_VIRTUAL_TERMINAL_PROCESSING (which is a feature of conhost, even without Terminal involved.)

This means that that circling behavior happens pretty much instantly. Like, after the first 25 lines are printed, the bottom line stops incrementing. It'll always be 25, or 30, or whatever the viewport height is.

1. footnote about terminal/console being the same

It's kinda funny actually, they also share code. It's kinda like this:

Terminal shares modules with conhost, and also runs a separate copy of conhost. The shared modules deal with text buffers, parsing VT, etc. because both of them have got to do it.

[DHowett]

If you really want to go crazy with doc reading, @miniksa wrote up our ecosystem roadmap which covers a good bit of our history and where we're taking this starship. :)