fix(web): better predictive-text handling of text wordbreaking transitions

[jahorton]

Fixes: #12494.
Replaces #12860.

The underlying issue was occurring due to a previously unforeseen consequence of default wordbreaker use, combined with some of our custom handling.

Good case: suppose the text just typed reads isn'. A natural suggestion for English is the word isn't. While end-of-token ' is cause for wordbreaking, since the text insertion point is adjacent to the ', we disable wordbreaking for that ' to facilitate the isn't suggestion.

Bad case: suppose we just typed text' and then add another character that should trigger wordbreaking - like a " - gets typed. At this point, due to our handling for the "good case", we transition from a tokenized context of [text'] to [text, ', "]. The "old context" no longer matches - [text'] vs [text, '] - but the method responsible for detecting this previously did not do so, which is what led to the original error.

Additionally, I added a unit test for this case to the testing suite. It checks the circumstance that leads to the error, rather than the original error itself, so user testing is still wise here.

User Testing

TEST_NOCRASH: follow the steps outlined in #12494 and verify that the crash no longer happens.

[keymanapp-test-bot]

User Test Results

Test specification and instructions

• ✅ TEST_NOCRASH (PASSED) (notes)

Test Artifacts

• Android
  • Keyman for Android apk
  • FirstVoices for Android 16.0 apk
  • FirstVoices Keyboards for Android apk
  • KeyboardHarness apk
  • Keyman for Android 16.0 apk
  • KMSample1 apk
  • KMSample2 apk
• Developer
  • Keyman Developer
  • Compiler Regression Tests
  • kmcomp.zip
• iOS
  • Keyman for iOS (simulator image)
  • FirstVoices Keyboards for iOS (simulator image)
  • TestFlight internal PR build version - 18.0.164 (0.12864.12510)
• Keyboards
  • Test Keyboards
• Web
  • KeymanWeb Test Home
• Windows
  • Keyman for Windows
  • FirstVoices Keyboards for Windows
  • Text Editor (32 bit)
  • Text Editor (64 bit)

[dinakaranr]

Test Results

I tested this issue with the attached "18.0.164-alpha-test-12864" build on the Andoird 14. Here is my observation.

• TEST_NOCRASH (Passed):

1. Open the keyman app in the Android mobile app.
2. Added EuroLatin keyboard.
3. Added a sentence.
4. Added letters to these orders.
5. Longpress g --> g̃,
6. Longpress h --> ḥ,
7. Longpress j --> ĵ,
8. Flick down on c --> ',
9. Flick down on v --> ",
10. Verified that no error appears on the toast message.
    It works well in good and bad cases. It works well. Thank you.