Soft keyboard input for iOS #3571

simlay · 2024-03-05T17:57:12Z

Tested on all platforms changed
Added an entry to CHANGELOG.md if knowledge of this change could be valuable to users
Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
Created or updated an example program if it would help users understand this functionality
Updated feature matrix, if new features were added or implemented

Related to #2260 and #1823.

kchibisov · 2024-03-05T18:12:20Z

src/platform/ios.rs

@@ -100,6 +100,7 @@ pub trait WindowExtIOS {
    ///
    /// The default is to not recognize gestures.
    fn recognize_rotation_gesture(&self, should_recognize: bool);
+    fn set_keyboard_visible(&self, visible: bool);


We already have set_ime_allowed, so wire to it for now.

The WinitTextViewDelegate must remain in memory or it will be collected on the objective-c side.

madsmtm

Have added a few comments wrt. objc2 stuff.

Regarding the implementation, I'm unsure if this is the right approach? Adding a hidden view like this seems like it could get into a lot of trouble with event focus?

There's no other way to control the software keyboard?

(I suspect this will be easier to solve after something like #3506, but I'd be fine with this until then).

src/platform_impl/ios/uikit/text_field.rs

madsmtm · 2024-03-15T00:10:59Z

src/platform_impl/ios/text_field.rs

+        // These are methods from UIResponder
+        #[method(becomeFirstResponder)]
+        pub fn focus(&self) -> bool;
+
+        #[method(resignFirstResponder)]
+        pub fn unfocus(&self) -> bool;


Should be moved to an extern_methods! on UIResponder.

src/platform_impl/ios/uikit/text_field.rs

madsmtm · 2024-03-15T00:18:05Z

src/platform_impl/ios/text_field.rs

+        fn window(&self) -> Option<Id<WinitUIWindow>> {
+            unsafe { msg_send_id![self, window] }
+        }


I'd rather have this on UIView and return UIWindow, and then when the WinitUIWindow is needed, you'd use the method on UIView, but with an Id::cast to turn it into WinitUIWindow.

madsmtm · 2024-03-15T00:18:09Z

src/platform_impl/ios/text_field.rs

+}
+
+declare_class!(
+    pub(crate) struct WinitTextField;


I suspect you don't need to subclass UITextField, i.e. this class is unnecessary as long as you make sure to store the WinitTextFieldDelegate somewhere else (e.g. on WinitView, as done currently).

Co-authored-by: Mads Marquart <[email protected]>

simlay · 2024-04-03T16:12:12Z

Regarding the implementation, I'm unsure if this is the right approach? Adding a hidden view like this seems like it could get into a lot of trouble with event focus?

After some research and thought, I don't think so but I'm not sure what the least-bad-way to do it is. I wrote about this in a chat but felt like it needed to on this issue.

I decided to look at how Flutter does it and how SwiftUI probably does it.

For Flutter: I created a simple flutter app with just a TextField, loaded it into my iOS simulator inspected the UI heirarchy in xcode, and it showed FlutterTextInputView which is defined in the depths of flutter/engine repo. It's unclear how one puts various UIViews (from the apple uikit framework) mixed in with skia rendering parts.

For SwiftUI, I created a simple view with a TextView and it's a TextEditorTextView in the UI hierarchy. This has some sub parts which are UIViews. From the relatively few results on a github search for TextEditorTextView, the one that fits is SwiftUI.TextEditorTextView : UITextView which say they're generated/reverse engineered with RuntimeBrowser. So, it seems that UITextView mixed in similar to flutter.

I'd look into how xamrin does text input but given how old it is, I'm pretty sure it's using a UITextView or the like as well.

lucasmerlin · 2024-05-16T09:20:20Z

In my fork of winit I implemented iOS keyboard input by just implementing UIKeyInput for the WinitView. This works pretty well, but it doesn't support autocomplete / autocorrect, for that to work we would need to implement UiTextInput as well.

Here are the relevant changes:

lucasmerlin@0eec676#diff-6540857eb4ce8f18ff73ad5a24513873e5572acb1ca867b055a462bdf8ea9e23

If this seems like a better way, I could open a PR

madsmtm · 2024-05-16T10:31:03Z

@simlay: Yeah, why are we not just using UITextInput and the related protocols? It really seems like UITextView/UITextField is the wrong approach here, since they also display the text itself, which we in Winit want to leave as a task to the user.

kchibisov · 2024-05-16T10:41:24Z

Isn't all the preedit, etc is shown as a part of the keyboard UI? If so, I don't see how it's different to Android where we want to do things like that.

If it shows in some bubble(which is likely not the case, because it's not desktop), then sure, but regular preedit/corrections is just an IME.

simlay · 2024-05-16T14:42:03Z

@simlay: Yeah, why are we not just using UITextInput and the related protocols? It really seems like UITextView/UITextField is the wrong approach here, since they also display the text itself, which we in Winit want to leave as a task to the user.

Oh that was just because I was thinking this was how other frameworks did it. I think someone on matrix pointed out that react-native, flutter et al implement the UITextInput protocols.

If this seems like a better way, I could open a PR

@lucasmerlin your changes seem more of the correct path forward. Like you said, the autocomplete part is tough. Does implementing the UITextInput protocols enable moving the cursor of the input?

lucasmerlin · 2024-05-16T15:41:22Z

@simlay

Does implementing the UITextInput protocols enable moving the cursor of the input?

No Idea, but looking at the docs, I think that's what the floating cursor apis might be for? I have no idea how this would be handled by winit though.

@lucasmerlin your changes seem more of the correct path forward.

Feel free to fork / modify / copy the code from my fork if you want to keep working on this. I think it might make sense to open a PR that implements UiKeyInput with basic text input support (which should be fairly straightforward based on my changes) and then open another one that implements UiTextInput afterwards.

madsmtm · 2024-08-19T00:33:51Z

I'm going to go with #3823 first, and then we can work from that to use the more feature-ful protocols to provide even better keyboard support. But thanks for starting on this work Sebastian!

Initial work for adding soft keyboard input

f9b058e

kchibisov reviewed Mar 5, 2024

View reviewed changes

Updated window example to set keyboard

78bac6b

simlay force-pushed the add-soft-keyboard-input-to-ios branch from 324d220 to 78bac6b Compare March 5, 2024 18:14

simlay and others added 6 commits March 5, 2024 13:31

Updates from comments

77d049c

cargo fmt

9ce9105

More changes

66c9658

Fixed WinitTextViewDelegate.

d3a54a2

The WinitTextViewDelegate must remain in memory or it will be collected on the objective-c side.

Minor changes

e276c06

Enabled the text to propigate to the main event loop

ab4a9e9

madsmtm reviewed Mar 15, 2024

View reviewed changes

simlay and others added 4 commits March 15, 2024 00:15

Update src/platform_impl/ios/uikit/text_field.rs

7200d84

Co-authored-by: Mads Marquart <[email protected]>

Update src/platform_impl/ios/uikit/text_field.rs

11728ba

Co-authored-by: Mads Marquart <[email protected]>

Update src/platform_impl/ios/uikit/text_field.rs

7bd5742

Co-authored-by: Mads Marquart <[email protected]>

minor updates from comments

d8094f1

madsmtm mentioned this pull request Apr 2, 2024

SoftInput (Software Keyboard) support #1823

Open

madsmtm added the DS - ios label May 5, 2024

lucasmerlin mentioned this pull request Jul 25, 2024

Basic iOS IME support #3823

Merged

5 tasks

madsmtm closed this Aug 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Soft keyboard input for iOS #3571

Soft keyboard input for iOS #3571

simlay commented Mar 5, 2024 •

edited by madsmtm

Loading

kchibisov Mar 5, 2024

madsmtm left a comment

madsmtm Mar 15, 2024

madsmtm Mar 15, 2024

madsmtm Mar 15, 2024

simlay commented Apr 3, 2024

lucasmerlin commented May 16, 2024

madsmtm commented May 16, 2024

kchibisov commented May 16, 2024

simlay commented May 16, 2024

lucasmerlin commented May 16, 2024 •

edited

Loading

madsmtm commented Aug 19, 2024

Soft keyboard input for iOS #3571

Soft keyboard input for iOS #3571

Conversation

simlay commented Mar 5, 2024 • edited by madsmtm Loading

kchibisov Mar 5, 2024

Choose a reason for hiding this comment

madsmtm left a comment

Choose a reason for hiding this comment

madsmtm Mar 15, 2024

Choose a reason for hiding this comment

madsmtm Mar 15, 2024

Choose a reason for hiding this comment

madsmtm Mar 15, 2024

Choose a reason for hiding this comment

simlay commented Apr 3, 2024

lucasmerlin commented May 16, 2024

madsmtm commented May 16, 2024

kchibisov commented May 16, 2024

simlay commented May 16, 2024

lucasmerlin commented May 16, 2024 • edited Loading

madsmtm commented Aug 19, 2024

simlay commented Mar 5, 2024 •

edited by madsmtm

Loading

lucasmerlin commented May 16, 2024 •

edited

Loading