dart:io - stdin does not report bytes for many keys on windows

[ellemenno]

Experience

i'm working on a cross-platform commandline app, testing with a simple stdin echo program in dart 2.16 (see below), and am noticing missing key codes when running in windows (but not in linux).

it seems like the silent keys on windows are Esc (0x1b), and the multi-byte code sequences that start with 0x1b on linux.

Expectation

i expect to see some key code for every non-modifier key on my keyboard. i recognize some code values may differ across platforms.

Notes

so far, i'm testing windows cmd and wsl ubuntu on the same windows 10 machine, using the same Terminal app, and i see character codes from linux for all non-modifier keys (except PrtScr and the Windows ⊞ key), but am getting no value for a number of important keys from windows (ESC, function keys, arrow keys, and Insert,Delete,Home,End,PgUp,PgDn).

update: added results from osx (practically identical to wsl)

Windows Terminal version 1.11.3471.0

win (cmd.exe)

> ver
Microsoft Windows [Version 10.0.19042.1466]
> dart --version
Dart SDK version: 2.16.0 (stable) (Mon Jan 31 15:28:59 2022 +0100) on "windows_x64"

wsl (ubuntu bash)

$ lsb_release --description
Description:    Ubuntu 18.04.5 LTS
$ dart --version
Dart SDK version: 2.16.0 (stable) (Mon Jan 31 15:28:59 2022 +0100) on "linux_x64"

OSX Terminal version 2.12 (443)

osx (darwin bash)

$ sw_vers
ProductName:    macOS
ProductVersion: 12.2
BuildVersion:   21D49
$ dart --version
Dart SDK version: 2.16.0 (stable) (Mon Jan 31 15:28:59 2022 +0100) on "macos_x64"

Test code

stdin.dart

import 'dart:io';

void main() {
  stderr.write('\n');
  stderr.write('type to echo the code sequence read by stdin.\n');
  stderr.write("type 'q' to exit.\n");

  stdin
    ..echoMode = false // for windows sake, echoMode must be disabled first
    ..lineMode = false; // see https://github.com/dart-lang/sdk/issues/28599#issuecomment-615940833

  var codeRead, codePrint;

  while (true) {
    codeRead = 0;
    codePrint = 0;
    while (codeRead <= 0) {
      codeRead = stdin.readByteSync();
    }
    if (codeRead == 0x7F) {
      codePrint = 0x2421; // unicode control picture for delete (U+2421)
    } else if (codeRead <= 0x20) {
      codePrint = 0x2400 + codeRead; // replace with matching control picture (U+2400 - U+2420)
    } else {
      codePrint = codeRead; // printable string, so use as is
    }
    stderr.write(
        "0x${codeRead.toRadixString(16).padLeft(2, '0')} '${String.fromCharCode(codePrint)}'\n");

    if (String.fromCharCode(codeRead) == 'q') {
      exit(0);
    }
  }
}

Results

table of characters reported by dart:io.stdin.readByteSync():

all same: 0x09 (␉), and printable chars 0x20 - 0x7e
Backspace 0x08 (␈) and Delete 0x7f (␡) are reported on all three OSes, but sometimes from different keys
Enter is reported as line feed 0x0a (␊) on linux and osx, and as carriage return 0x0d (␍) on win

F11 is skipped in the table, because the terminal apps consume it first to toggle full-screen or mission control, and it doesn't get passed along.

key	wsl & osx	win
ESC	0x1b '␛'
F1	0x1b '␛'
	0x4f 'O'
	0x50 'P'
F2	0x1b '␛'
	0x4f 'O'
	0x51 'Q'
F3	0x1b '␛'
	0x4f 'O'
	0x52 'R'
F4	0x1b '␛'
	0x4f 'O'
	0x53 'S'
F5	0x1b '␛'
	0x5b '['
	0x31 '1'
	0x35 '5'
	0x7e '~'
F6	0x1b '␛'
	0x5b '['
	0x31 '1'
	0x37 '7'
	0x7e '~'
F7	0x1b '␛'
	0x5b '['
	0x31 '1'
	0x39 '8'
	0x7e '~'
F8	0x1b '␛'
	0x5b '['
	0x31 '1'
	0x39 '9'
	0x7e '~'
F9	0x1b '␛'
	0x5b '['
	0x32 '2'
	0x30 '0'
	0x7e '~'
F10	0x1b '␛'
	0x5b '['
	0x32 '2'
	0x31 '1'
	0x7e '~'
F12	0x1b '␛'
	0x5b '['
	0x32 '2'
	0x34 '4'
	0x7e '~'
Insert	0x1b '␛'
	0x5b '['
	0x32 '2'
	0x7e '~'
Delete	0x1b '␛'
	0x5b '['
	0x33 '3'
	0x7e '~'
Arrow ↑	0x1b '␛'
	0x5b '['
	0x41 'A'
Arrow ↓	0x1b '␛'
	0x5b '['
	0x42 'B'
Arrow →	0x1b '␛'
	0x5b '['
	0x43 'C'
Arrow ←	0x1b '␛'
	0x5b '['
	0x44 'D'
PgUp	0x1b '␛'
	0x5b '['
	0x35 '5'
	0x7e '~'
PgDn	0x1b '␛'
	0x5b '['
	0x36 '6'
	0x7e '~'
End	0x1b '␛'
	0x5b '['
	0x46 'F'
Home	0x1b '␛'
	0x5b '['
	0x48 'H'

[ellemenno]

hi, @kevmoo it's been a year, and this issue still exists on the latest stable dart (3.3.2):

> dart --version
Dart SDK version: 3.3.2 (stable) (Tue Mar 19 20:44:48 2024 +0000) on "windows_x64"

Esc (0x1b), and the multi-byte code sequences that start with 0x1b enumerated in the table above (like arrow keys) aren't read by dart:io.stdin.readByteSync() on windows.

any chance this can be addressed in a near-term release?

[kevmoo]

Maybe @brianquinlan has some ideas?

[ellemenno]

hi @brianquinlan & @kevmoo, i'm out of my depth here, but i'd like to help move this forward if i can (please let me know if there is something specifically helpful that I could do..)

i've found that the ConEmu project has a handy demo utility called KeyEvents that shows what data is received on Windows when various keys are pressed. it demonstrates reacting to all the keys that Dart is missing in this issue.

comparing implementations, Dart's ReadByte for Windows function uses the ReadFile Windows API, while ConEmu's KeyEvents demo uses the ReadConsoleInput API.

looking at the Windows API docs, the limitations noted in this issue seem to be expected when using the ReadFile method (my emphasis):

To get keyboard input, a process can use ReadFile or ReadConsole with a handle to the console's input buffer, or it can use ReadFile to read input from a file or a pipe if STDIN has been redirected. These functions only return keyboard events that can be translated into ANSI or Unicode characters. The input that can be returned includes control key combinations. The functions do not return keyboard events involving the function keys or arrow keys. Input events generated by mouse, window, focus, or menu input are discarded.
_{High-Level Console Input and Output Functions}

bool Stdin::ReadByte(intptr_t fd, int* byte) {
  HANDLE h = GetStdHandle(STD_INPUT_HANDLE);
  uint8_t buffer[1];
  DWORD read = 0;
  BOOL success = ReadFile(h, buffer, 1, &read, nullptr);
  if (!success && (GetLastError() != ERROR_BROKEN_PIPE)) {
    return false;
  }
  *byte = (read == 1) ? buffer[0] : -1;
  return true;
}

_{stdio_win.cc}

..whereas the ReadConsoleInput API does return keyboard events for function and arrow keys, and escape, and the others that are missing keycodes in the chart above.

  // ..
  INPUT_RECORD r, rl = {};
  HANDLE h = GetStdHandle(STD_INPUT_HANDLE);
  HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE);
  SetConsoleCtrlHandler(HandlerRoutine, TRUE);
  BOOL lbEscPressed = FALSE;
  wchar_t szFormat[1024];
  // ..
  while (TRUE)
  {
    memset(&r, 0, sizeof(r)); dw = 0;
    if (ReadConsoleInput(h, &r, 1, &dw))
    {
      // ..
      __INPUT_RECORD_Dump(&r, szFormat);
      DWORD nLen = wcslen(szFormat);
      WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), szFormat, nLen, &nLen, NULL);
      // ..
    }
  }

_{KeyEvents.cpp}

so, can the Windows ReadByte implementation be updated to use the ReadConsoleInput API and gain parity with the Mac and Linux implementations?

[jtmcdole]

Just to note; mason_logger uses package win32 to enable raw mode. I use this in my own code to read arrow keys, backspace, and more from stdin:

        ///----- Arrow keys -----\\\
        case 0x41:
          // \1b[A
          final modifiers = rolling.length == 1
              ? Modifiers.lut[rolling[0] - 1]!
              : Modifiers.lut[0]!;

          _events.add(KeycodeEvent('↑', Keycode.up, modifiers: modifiers));
          return;

        case 0x42:
          // \1b[B
          final modifiers = rolling.length == 1
              ? Modifiers.lut[rolling[0] - 1]!
              : Modifiers.lut[0]!;
          _events.add(KeycodeEvent('↓', Keycode.down, modifiers: modifiers));
          return;

        case 0x43:
          // \1b[C
          final modifiers = rolling.length == 1
              ? Modifiers.lut[rolling[0] - 1]!
              : Modifiers.lut[0]!;
          _events.add(KeycodeEvent('→', Keycode.right, modifiers: modifiers));
          return;

        case 0x44:
          // \1b[D
          final modifiers = rolling.length == 1
              ? Modifiers.lut[rolling[0] - 1]!
              : Modifiers.lut[0]!;
          _events.add(KeycodeEvent('←', Keycode.left, modifiers: modifiers));
          return;