Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spec(web): Keyman Core integration for Web #11293

Open
1 task done
ermshiperete opened this issue Apr 23, 2024 · 1 comment
Open
1 task done

spec(web): Keyman Core integration for Web #11293

ermshiperete opened this issue Apr 23, 2024 · 1 comment
Assignees
Milestone

Comments

@ermshiperete
Copy link
Contributor

ermshiperete commented Apr 23, 2024

Introduction

Currently Keyman Core is only used on the desktop platforms (Windows, Mac, Linux). Keyman Core supports KMN as well as LDML keyboards.

Keyman Engine for Web has a separate implementation that only supports KMN based keyboards. This is also used by Keyman for Android and Keyman for iOS.

Keyman Engine for Web currently doesn't use the .kmx keyboard files but .js files that get generated by kmc as well.

Keyman Core doesn't deal with touch and OSK aspects of a keyboard.

LDML keyboards use an enhanced version of KMX (KMX+).

We would like to integrate Keyman Core in Keyman Engine for Web.

Architecture

  • Keyman Core API will be accessed through WASM
  • OSK will continue to be handled by Keyman Engine for Web
  • Keyman Engine for Web will read a subset of KMX+ for touch layout/OSK
  • Keyboards in .js format will be deprecated (probably v19). Instead we use .kmx everywhere.

Implementation

Refactorings/Cleanup in Keyman Engine for Web (#11374)

  • move gesture-recognizer, input-processor, keyboard-processor from common to web/src (do we still need {input,keyboard}-processor in the future or will this be replaced by Core?)

  • for clarity, rename KeymanEngine.core to KeymanEngine.inputProcessor

  • rename keyboardCache to keyboardStore (and StubAndKeyboardCache to StubAndKeyboardStore?)

  • chore(web): restructure repo #11374

Changes required to Keyman Engine for Web

  • check backspace code - nothingDeletable - make sure LDML's return is understood by the K_BKSP fixup code
  • probably want a new subclass of KeyboardHarness (or KeyboardInterface, or in-between) for LDML
    • add interface KeyboardHarness -> AbstractKeyboardInterface -> KeyboardInterface
    • add new class CoreKeyboardInterface
  • KeyboardProcessor needs to know when to load CoreKeyboardInterface
    1. set KeyboardProcessor.activeKeyboard called by another layer
    2. caller will need to be able to load a .kmx instead of a .js

KMX Loader

Keyman Engine for Web will load .kmx files instead of .js. The .kmx loader should detect at load time if first bytes are KXTS and treat it as kmx, otherwise eval() as a script reject it and it will be loaded by the existing .js loader.

  • Onscreen keyboard - and layer ids will need some wrangling.
    • setNumericLayer
    • should be generic intent stuff

Changes required to Core API

  • core API will need to have a concept of layerId

Related issues and discussions

@ermshiperete ermshiperete added this to the 18.0 milestone Apr 23, 2024
@mcdurdin
Copy link
Member

The .kmx loader should detect at load time if first bytes are KXTS and treat it as kmx, otherwise eval() as a script.

I don't think we want to do sniffing and eval as script. The pathway for loading a .js keyboard right now is injecting as a <script> element. The new pathway will take a blob from a source tbd and should just reject it if it doesn't start with the KMX magic bytes KXTS.

@mcdurdin mcdurdin modified the milestones: 18.0, A18S8 Apr 29, 2024
@darcywong00 darcywong00 modified the milestones: A18S8, A18S9 Aug 17, 2024
@mcdurdin mcdurdin added this to Keyman Aug 19, 2024
@ermshiperete ermshiperete moved this to Todo in Keyman Aug 19, 2024
ermshiperete added a commit that referenced this issue Aug 20, 2024
- add temporary function to Core for this POC
- add new CoreProcessor to access Keyman Core WASM
- add unit tests for new core processor
- add code to KeymanEngine and InputProcessor to load the new CoreProcessor
- add web server to manual tests and new action `start` to build script

This change requires the manual tests to be loaded from a web server
instead of loaded as file, because otherwise the wasm code won't be
loaded.

Currently we always load CoreProcessor. This should be improved in a future
change to only load when it is actually needed.

Part-of: #11293
ermshiperete added a commit that referenced this issue Aug 20, 2024
- add temporary function to Core for this POC
- add new CoreProcessor to access Keyman Core WASM
- add unit tests for new core processor
- add code to KeymanEngine and InputProcessor to load the new CoreProcessor
- add web server to manual tests and new action `start` to build script

This change requires the manual tests to be loaded from a web server
instead of loaded as file, because otherwise the wasm code won't be
loaded.

Currently we always load CoreProcessor. This should be improved in a future
change to only load when it is actually needed.

Part-of: #11293
ermshiperete added a commit that referenced this issue Aug 20, 2024
- add temporary function to Core for this POC
- add new CoreProcessor to access Keyman Core WASM
- add unit tests for new core processor
- add code to KeymanEngine and InputProcessor to load the new CoreProcessor
- add web server to manual tests and new action `start` to build script

This change requires the manual tests to be loaded from a web server
instead of loaded as file, because otherwise the wasm code won't be
loaded.

Currently we always load CoreProcessor. This should be improved in a future
change to only load when it is actually needed.

Part-of: #11293
ermshiperete added a commit that referenced this issue Aug 20, 2024
- add temporary function to Core for this POC
- add new CoreProcessor to access Keyman Core WASM
- add unit tests for new core processor
- add code to KeymanEngine and InputProcessor to load the new CoreProcessor
- add web server to manual tests and new action `start` to build script

This change requires the manual tests to be loaded from a web server
instead of loaded as file, because otherwise the wasm code won't be
loaded.

Currently we always load CoreProcessor. This should be improved in a future
change to only load when it is actually needed.

Part-of: #11293
ermshiperete added a commit that referenced this issue Aug 20, 2024
- add temporary function to Core for this POC
- add new CoreProcessor to access Keyman Core WASM
- add unit tests for new core processor
- add code to KeymanEngine and InputProcessor to load the new CoreProcessor
- add web server to manual tests and new action `start` to build script

This change requires the manual tests to be loaded from a web server
instead of loaded as file, because otherwise the wasm code won't be
loaded.

Currently we always load CoreProcessor. This should be improved in a future
change to only load when it is actually needed.

Part-of: #11293
ermshiperete added a commit that referenced this issue Aug 30, 2024
- loading a keyboard from a BLOB
- getting the on-screen keyboard layout from Core. This is an internal-
  only API because of it's use of C++.

Part-of: #11293
Part-of: #8093
@darcywong00 darcywong00 modified the milestones: A18S9, A18S10 Aug 31, 2024
@darcywong00 darcywong00 modified the milestones: A18S10, A18S11 Sep 14, 2024
ermshiperete added a commit that referenced this issue Sep 23, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 23, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 23, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 23, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 23, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 24, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 24, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 24, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 24, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 24, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 24, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
ermshiperete added a commit that referenced this issue Sep 24, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Part-of: #11293
@darcywong00 darcywong00 modified the milestones: A18S11, A18S12 Sep 28, 2024
@darcywong00 darcywong00 modified the milestones: A18S12, A18S13 Oct 11, 2024
@darcywong00 darcywong00 modified the milestones: A18S13, A18S14 Oct 26, 2024
@darcywong00 darcywong00 modified the milestones: A18S14, A18S15 Nov 9, 2024
@darcywong00 darcywong00 modified the milestones: A18S15, A18S16 Nov 24, 2024
ermshiperete added a commit that referenced this issue Nov 27, 2024
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa32
Cherry-Pick-Commit: 59019cc
Cherry-Pick-Commit: bc46458
Cherry-Pick-Commit: d06aa29
Cherry-Pick-Commit: 1c88166
Cherry-Pick-Commit: 069cd21
Cherry-Pick-Commit: 052ae2e
Cherry-Pick-Commit: 11a2a3b

Part-of: #11293
Part-of: #8093
@darcywong00 darcywong00 modified the milestones: A18S16, A18S17 Dec 7, 2024
@darcywong00 darcywong00 modified the milestones: A18S17, A18S18 Dec 21, 2024
@darcywong00 darcywong00 modified the milestones: A18S18, A18S19 Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

No branches or pull requests

3 participants