Keystrokes All "Unidentified" On Android

[gjhami]

Wow, this one was a rabbit hole:

Summary

Some platforms, including Android, return a keyCode and code value of 229 ("Unidentified") when keystrokes are typed but composition is not yet complete, ex. there's a proposed autocorrect or autocomplete value displayed. In practice, this means keystrokes don't appear on the user's screen as they type when they visit a cuddlephish page, because cuddlephish doesn't send any valid individual keystrokes through the web socket as they're typed. This issue did not affect iOS or desktop at the time of writing.

Background

All keystrokes are captured as "Unidentified" (keyCode 229) in the keylog when a cuddlephish site is visited on Android with a standard keyboard layout, but the keylog works fine on desktop and iOS. After going back and forth and researching this issue several times, I believe I've found the root cause explained well in this slate issue. Essentially it is a quirk of Android, and possibly other platforms, caused by the use of Input Method Editors (IMEs). IMEs traditionally related to characters that could only be typed through multiple keystrokes but is now comingled with autocorrect/autocomplete and involves a lot of intricacies regarding when it sends keystrokes. Here's a great tool for visualizing what events are generated on various platforms such as Android during typing.

Existing Solutions

• Some (in my opinion fairly hacky) solutions have been presented (1), that rely on writing an event listener to check the character code of the last value in a text box every time an input or beforeinput event is fired. This does not account nicely for the use of backspaces or editing the value of the input anywhere besides the last character.
• Mozilla's solution is to discard all keydown events where the keyCode is 229 or IME composition is in progress. This doesn't work, because cuddlephish needs to capture and send keystrokes to the websocket for display to the user as they're typed, not when composition is finished.

Proposed Solution

• Ideal: An ideal solution would involve sending individual keystrokes through the websocket while also updating the full value of the target input field in the cuddlephish server's browser instance when IME composition concludes, such as if an autocorrect or autocomplete value is selected. I believe this would involve extracting information about input fields on the cuddlephish server's browser page and associating them with hidden input fields on the page presented to the user, then creating event listeners that fire when the user's input fields are updated to sync their text contents with the ones in the cuddlephish server's browser. I believe this may involve heavy lifting with puppet beyond my expertise and a poor implementation could negatively impact performance in a way that's visible to the user.
• Realistic: When keyCode 229 is observed, go with the hacky solution and just grab the character code of the last character typed in a hidden input field that is focused on when a user clicks the video div. When keyCode 229 is not observed, continue to observe and send keystrokes using the current implementation. This would have the added bonus of also fixing the issue where keyboards aren't presented on mobile.

I am happy to submit a PR for the realistic solution in the near future.