Japanese character output in Integrated Terminal of Visual Studio Code causes the next prompt out of position after upgrading to Creators Update

[yusuke-konishi]

• A brief description

Japanese character output in Integrated Terminal of Visual Studio Code causes the next prompt out of position after upgrading to Creators Update. This issue is reproducible 100%.

For example, echo あ causes the next prompt out of position. The screenshot below is the result of ls -la with en only, and echo あ. ls -la does not cause any problem, but echo あ causes the next prompt out of position.

• Expected results

Japanese character output in Integrated Terminal of Visual Studio Code keeps the next prompt in the correct position.

• Actual results (with terminal output if applicable)

Japanese character output in Integrated Terminal of Visual Studio Code causes the next prompt out of position.

• Your Windows build number

Version 1703, build 15063.13 (Japanese)

• Steps / All commands required to reproduce the error from a brand new installation

1. Install Windows 10 Creators Update
2. Install BoW on Creators Update
3. Install Visual Studio Code (v1.11.1)
4. Open BoW in Integrated Terminal by [Ctrl] +[@]
5. Run echo あ
   -> This causes the next prompt out of position.

• Other troubleshooting

I tried repro on the following three environment by Visual Studio Code v1.11.1. By this results, this does not seem to depend on Visual Studio Code, but Creators Update. Can't BoW handle this issue?

1. OK
   OS: build 14393.0
   BoW: Ubuntu 14.04

2. NG
   OS: build 15063.13 (upgraded from build 14393.0)
   BoW: Ubuntu 14.04 (not upgraded)

3. NG
   OS: build 15063.13
   BoW: Ubuntu 16.04

[fpqc]

I think this is a known issue right now in the console that was not expectedto be fixed until RS3 (support for 2wide characters). There's another thread that mentions this in the case of Chinese.

Edit:Nevermind, support was added in bash, but not in the WinPTY project which is a kind of hackish thing used to attach the console to VSCode

[dori4n]

Actually, this issue is even worse, if your system locale (also referred to as language for non-Unicode programs) isn't one which supports the character set. On a Japanese system this would be Shift-JIS, on a Chinese one GB18030, and ISO-2022-KR on a Korean one, which are mostly not compatible with each other. Changing the system locale requires a reboot to apply. Any characters not included in the character set will show up as question marks (despite font support) and you cannot type them into the terminal (any terminal, Powershell/Command Prompt included) even after setting the application code page to 65000 or 65001 using any input method editor or by copy and pasting the characters from the clipboard. This is massively annoying. Terminal emulation has been messed with to great extent just for Bash compatibility, but this, you actually completely managed to break. Congrats... 😐
PS: It also breaks new-lines for commands which exceed the width of the terminal display.

@fpqc: No, support was not added in bash.exe. WinPTY just passes through what comes back from the console that is mapped to. All of Windows' terminal emulators are broken.

[zadjii-msft]

I think this is a WinPTY issue, not a conhost one:

Looks like rprichard/winpty#105 to me.

[fpqc]

@zadjii-msft Haha that's exactly the test I did, and it was working properly. Mouse selection even works right.

[bitcrazed]

Closing since this is a very old issue that's likely fixed by now, and is a VSCode/WinPty issue that @Tyriar may have insight into.

[Tyriar]

I remember there was an issue like this, pretty sure it was fixed shortly afterwards.