SoftInput (Software Keyboard) support

[semtexzv]

On android, the application should be able to request showing & hiding of soft input.

This is available through ndk_glue for android.

But this strangely feels like functionality that should be attached to window.
eg. for desktop we have cursor grabbing, for mobile, having methods for showing keyboard seems similar.

[torokati44]

Yeah, this would be really useful. Especially since so far I have been completely unable to pop up the virtual keyboard on winit's NativeActivity.
I does not appear when addingandroid:windowSoftInputMode="stateAlwaysVisible|adjustResize" to the <activity> entry in AndroidManifest.xml, nor when calling ndk_glue::native_activity().show_soft_input(true); from Rust.

[Fancyflame]

So... does any guy have any ideas? ndk_glue::native_activity() is deprecated, it says using ndk_context::android_context().vm() instead, but what next? I'm a noob at Android programming.

[msiglreith]

Not sure what's up with the NDK function but here is an example going trough jni: https://github.com/msiglreith/pixels-android/blob/master/src/lib.rs#L30 which shows the keyboard when tapping the screen.
It uses current main branch as virtual keycode have just been added recently for Android, otherwise it will just return empty ):

[Fancyflame]

This example can work on my device. But do you mean this method has a compatibility problem?

[msiglreith]

Sry, I referred to the Android winit platform implementation not Android itself: #2226
Should work with older Android versions as well

[Fancyflame]

Thanks. That's okay if it only needs update winit to the latest version. 😃
To be honest, I cannot understand what you wrote due to my lack of knowledge about Android and Java.... Can I copy your implementation to my project? I'll mark your repo.

[Fancyflame]

@msiglreith Then, how to hide it?

[msiglreith]

Can I copy your implementation to my project?

Feel free. I will probably look into opening a PR for this in the next days if there are no further issues with the JNI implementation.

Then, how to hide it?

I pushed a small change in the meantime to the repository from above which implements it (hides when tapping the screen again).

[Fancyflame]

@msiglreith Thanks for your example. I'm sorry for bothering you again, but... I have another two questions:

• How to popup and hide safe keyboard on Android for password input, just like <input type="password"> does in HTML?
• How to prevent the IME from popping up sometimes when press keys on PC (Windows 10)? (Sorry if not related to this issue)

[msiglreith]

How to popup and hide safe keyboard on Android for password input, just like does in HTML?

I think that's more tricky and not something which can be easily implemented. I would assume creating a custom View which will set the corresponding input type but that requires interfacing/compiling Java, which would require support on the tooling front in general...

How to prevent the IME from popping up sometimes when press keys on PC (Windows 10)? (Sorry if not related to this issue)

I haven't tried IME myself - you might want to look at #2243 which seems to add more functionality for it (in particular enabling/disabling IME)

[ElhamAryanpur]

are there any plans for implementation? workarounds posted here doesn't work anymore either

[madsmtm]

There's been some progress on iOS in #3571, and some progress on Android in #2993, and finally we've been having some discussion with the Xilem folks about it. I think there are vague plans to work on it in/post v0.31, but that's all I can say.

[kchibisov]

I'd add that if people are willing to put work into it, I'm all for having it. It's just should be just a little bit consistent with the rest of IME we have around, but that's about it.

[ElhamAryanpur]

I see, I'd love to help but I honestly don't know anything about JNI. will look further to see if theres some parts I can contribute to. Thank you!

[mvvvv]

I tested the Android keyboard launched via JNI on the Meta Quest2. It works exactly like any Android device.
https://github.com/mvvvv/StereoKit-rust/blob/master/src/tools/os_api.rs#L191

Indeed winit 0.30 blocks many characters including accented characters which it replaces completely with a backspace and erases the character typed previously. I made a short video: https://youtu.be/os8cucfF3D8

If you need to test I have an environment for that.

[daxpedda]

Web can do this via the VirtualKeyboard API, only supported in Chromium.

Cc @will-hart, Winit will implement a fallback using a hidden text input as well, unless I find some better method for non-Chromium browsers.

[jancespivo]

I've created PR in android-activity rust-mobile/android-activity#178 could you check it please?

cc @podusowski @xorgy

[mvvvv]

About this PR which is what I'm using for OpenXR/StereoKit-rust, may I suggest to enhance the KeyAction Enum with a more complete list. This will avoid the following warnings:

12-30 15:16:17.601 23830 23830 W ViewRootImpl[NativeActivity]: Dropping event (no window focus):KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_DEL, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=0, downTime=0, deviceId=-1, source=0x0, displayId=0 }
12-30 15:16:17.601 23830 23830 W ViewRootImpl[NativeActivity]: Cancelling event (no window focus):KeyEvent { action=ACTION_UP, keyCode=KEYCODE_DEL, scanCode=0, metaState=0, flags=0x20, repeatCount=0, eventTime=0, downTime=0, deviceId=-1, source=0x0, displayId=0 }
12-30 15:16:17.601 23830 23830 W ViewRootImpl[NativeActivity]: Cancelling event (no window focus):KeyEvent { action=ACTION_UP, keyCode=KEYCODE_DEL, scanCode=0, metaState=0, flags=0x20, repeatCount=0, eventTime=0, downTime=0, deviceId=-1, source=0x0, displayId=0 }
12-30 15:16:17.602 23830 23830 W ViewRootImpl[NativeActivity]: Dropping event (no window focus):KeyEvent { action=ACTION_MULTIPLE, keyCode=KEYCODE_UNKNOWN, scanCode=0, characters="ò", metaState=0, flags=0x0, repeatCount=0, eventTime=19026369, downTime=19026369, deviceId=-1, source=0x101, displayId=-1 }
12-30 15:16:17.605 23830 23882 D SK_rust_Demos: !!!KeyboardInput: KeyEvent { physical_key: Code(Backspace), logical_key: Named(Backspace), text: None, location: Standard, state: Released, repeat: false, platform_specific: KeyEventExtra }

And will give access to ACTION_MULTIPLE which is the only way to get enhanced characters(here ò), smileys and utf8.

[jancespivo]

@mvvvv Yes It is the same approach I saw in your post. I tried the windowInsets one as well however it hasn't work.

The missing enhanced characters in keyEvent is completely orthogonal problem and definitely something I would like to solve. Could we move the conversation to the related issue? Ping me there please :)

[mvvvv]

Done #4067