Incorrect line ending values in wsl console output

[nickcampau]

Windows Version

10.0.19044.2604

WSL Version

1.1.3.0

Are you using WSL 1 or WSL 2?

• WSL 2
• WSL 1

Kernel Version

5.15.90.1

Distro Version

No response

Other Software

PowerShell 7.3.3

Repro Steps

I was having difficulties parsing the output from any wsl command with PowerShell without a bunch of additional lines getting added to the output if capturing the output into an array.
Here is an example output from the version command.

❯ wsl --version
WSL version: 1.1.3.0
Kernel version: 5.15.90.1
WSLg version: 1.0.49
MSRDC version: 1.2.3770
Direct3D version: 1.608.2-61064218
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.19044.2604

Array capture output:

❯ (wsl --version)
WSL version: 1.1.3.0

Kernel version: 5.15.90.1

WSLg version: 1.0.49

MSRDC version: 1.2.3770

Direct3D version: 1.608.2-61064218

DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp

Windows version: 10.0.19044.2604


❯ (wsl --version).Length
15

I saved the commands output into a file (i.e. wsl --version > file.txt) and discovered the cause is due to an incorrect character values placed at the end of each line.

The output is little endian utf16 format and each line is separated with \u0D00 + \u00A0 character sequence. If you look closely you'll notice that \u0D00 should be \u00D0 instead. 

Expected Behavior

wsl output is in a standard character encoding with appropriate line ending character values.

Actual Behavior

Output contains an invalid line ending sequence.

Diagnostic Logs

No response

[nickcampau]

To work around this issue the array output can be filtered with the following command

(wsl --version | ? {$_.Replace("`0","")})

[OneBlue]

Thanks for reporting this @nickcampau. This is because wsl writes to stdout in unicode by default. You can work around this by setting the environment variable WSL_UTF8 to 1, which wil force all output to be in UTF8.

[nickcampau]

@OneBlue thanks for the tip on enabling utf-8 output, that avoids the line ending bug I was getting.

However, I think you missed what exactly the bug was that I reported. I understand the the (default) output from wsl is Unicode (utf-16 to be exact) and that isn't an issue. What is wrong is one of the character values at the end of each line. Instead of outputting a carriage return value (\u00D0) the character (\u0D00 MALAYALAM SIGN COMBINING ANUSVARA
ABOVE) is used instead. To put it another way, the incorrect byte order of the value for the carriage return value was outputted.

This issue as reported has not been resolved and this has been marked closed. Can you reopen this issue?

[OneBlue]

Thanks @nickcampau. I can't reproduce this though. Can you explain how you got that unicode character from wsl and how you computed its value ?

[nickcampau]

@OneBlue I ran the following command in my PowerShell (ver 7.3.3.) console. wsl --version > wsl_version.txt and opened the file in a hex editor. I ran the same command in a cmd console and that file was encoded correctly as a UTF-16 LE file. So there looks like there is a mishandling of the output only with PowerShell.