Better support for touch interfaces

[EricRabil]

Hi there! I'm a big fan of Theia and it's potential, especially as a powerful IDE replacement for tablets (such as my iPad Pro). After using Theia on my iPad for a day, I can already see some weak points in terms of usability with a touch interface and mobile accessories.

• Arrows on my iPad Smart Keyboard do not work at all in neither the Terminal nor the Monaco shell. (https://www.grapecity.com/en/forums/wijmo/arrow-keys-dont-work-on-ip)

• Scrolling is a mess due to overscrolling. I had to lightly mod my Theia instance to disable overscrolling.

• Context menus are not dismissible without reloading the page.

• There is no way to resize panels in the IDE from a touch interface.

• The page is very zoomed in because there is no meta-tag controlling the viewport initials size (https://developer.mozilla.org/en-US/docs/Mozilla/Mobile/Viewport_meta_tag)

Theia already serves as an excellent IDE on tablets, and I would love to see it get even better. I would love to contribute to the project and help improve usability on touch interfaces after I get more acquainted with the codebase and guidelines.

Patch List

• Viewport Tag
• Overscroll

[svenefftinge]

Better support for touch would be very welcome. Let us know if you need help in getting started.

[sr229]

Hey @EricRabil even if we can get the interface to work, Monaco does not support touch natively AFAIK, so this would be useless if we can't use Monaco on iPad.

[svenefftinge]

But there seem to be 'hacks' available that could be considered until someone solves it more holistically.
See here for reference: microsoft/monaco-editor#293

[EricRabil]

@sr229 while you are correct in that Monaco does not support native touch, we (Theia) can still do our part to make it as usable as possible. Modifications that I made to my local Theia instance have made using Theia so much easier, even without changes to Monaco itself.

[jankeromnes]

@EricRabil It's really cool that you've already looked into this! Could you please share some info on your local Theia modifications? (E.g. uploading some diff or braindump somewhere would be perfect.)

I'd love to see if I can help improve Theia on touch devices too (I do a lot of dev work on my smartphone and sometimes on my iPad). Maybe we can team up on this?

[sr229]

FWIW I think it's best we upstream any changes we can make for Monaco so everyone can benefit (including VSCode)

[jankeromnes]

@EricRabil Ping -- how were you able to fix Theia's overscrolling on iPad? 🙂

I had to lightly mod my Theia instance to disable overscrolling.

I found this post that suggests a combination of:

html, body {
  overflow: hidden;
}

and

targetElement.ontouchend = (e) => {
  if (/* targetElement is scrolled to top or bottom */) {
    e.preventDefault();
  }
};

Was your fix something like that?

[EricRabil]

Hi, I believe those CSS declarations fixed overscrolling for me. I very hackily fixed issues as I stumbled across them, but have not modified Theia since I opened my ticket. If I have time this weekend I will sift through my Theia instance and report back with my patches.

On Fri, May 10, 2019 at 4:16 AM Jan Keromnes ***@***.***> wrote: @EricRabil <https://github.com/EricRabil> Ping -- how did you fix Theia's overscrolling on iPad? I had to lightly mod my Theia instance to disable overscrolling. I found this post <https://medium.com/jsdownunder/locking-body-scroll-for-all-devices-22def9615177> that suggests a combination of: html, body { overflow: hidden; } and targetElement.ontouchend = (e) => { if (/* targetElement is scrolled to top or bottom */) { e.preventDefault(); } }; Was your fix something like that? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3557 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB5N7BODXLODBBZU6Y47EZDPUUVMTANCNFSM4GFONRUA> .

-- --Eric Rabil (iPhone)

[jankeromnes]

Thanks a lot @EricRabil! That would be very helpful.

(Also, please don't bother cleaning up the patches, I'm perfectly happy to sift through a giant diff. 🙂)

[EricRabil]

I have a long flight with an even longer layover tonight, I’ll see what a I can muster together. I’ll fork the repo and put all my changes there.

On Fri, May 17, 2019 at 5:44 AM Jan Keromnes ***@***.***> wrote: Thanks a lot @EricRabil <https://github.com/EricRabil>! That would be very helpful. (Also, please don't bother cleaning up the patches, I'm perfectly happy to sift through a giant diff. 🙂) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3557>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB5N7BPLEZU2NUYTCQJQ4PDPVZ47HANCNFSM4GFONRUA> .

-- --Eric Rabil (iPhone)

[EricRabil]

Regarding overscrolling, there is no JavaScript required. I fixed it in my fork by modifying the .theia-ApplicationShell class as follows:

.theia-ApplicationShell {
  position: fixed; /** keeps element in place (necessary for MobileSafari) **/
  overflow: hidden; /** disables overflow **/
  top: 0;
  left: 0;
  right: 0;
  bottom: 0;
  background: var(--theia-layout-color0);
}

[EricRabil]

Regarding arrow keys on the smart keyboard, you can detect most (if not all) key events that are not currently fired by listening for document.addEventListener('selectionchange'). The only caveat is that there must be some kind of value in the textarea.inputarea element within Monaco, otherwise there is no selection to be changed. I'm gonna investigate how I can hack this in without altering functionality.

[EricRabil]

Great news! In iPadOS 13b1, arrow key functionality is fixed. Apple has given MobileSafari on the iPad a desktop-class browsing experience, and as such, there is full keyboard access. I am able to use my arrow keys in any Monaco-based editor now.

[jankeromnes]

Hi @EricRabil! Could you please edit your first comment, and make each item a checkbox? (You can simply start each item line with - [ ], or - [x] if the item is already addressed.)

This will allow us to more easily see which items are fixed, and which ones remain to address. 🙂

[vartanbeno]

I noticed two things:

1. The inability to dismiss context menus, which is tracked in Context menus are not dismissible on iPad (iOS 12) #6322.
2. I can open up multiple context menus at once if I do multiple long presses.

[jankeromnes]

@vartanbeno Thank you for reporting these!

Which version of iOS are you using? I was also seeing these issues all the time, but since a recent iOS upgrade on my iPad (to iOS 13 I think), context menus are now always dismissed when I tap outside of them (thus I can't open multiple context menus).

On my Android Pixel phone, I also don't see these problems anymore, so maybe they're platform/browser-specific.

[paul-marechal]

While the idea is interesting on its own, no active contributor currently has interest in implementing and maintaining such feature.

Were this to be supported, there's also the problem of preventing regressions: We don't have anyone with a touch surface to test, and ideally we would like an automated testing solution which I don't know if it exists for touch devices.

If anyone wants to champion this work, please come discuss at the dev meeting!