Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Strange characters inserted when trying to cut/paste #20

Closed
thtas opened this issue Jun 6, 2023 · 18 comments
Closed

[BUG] Strange characters inserted when trying to cut/paste #20

thtas opened this issue Jun 6, 2023 · 18 comments
Assignees
Labels
bug Something isn't working

Comments

@thtas
Copy link

thtas commented Jun 6, 2023

Bare metal or virtual machine:  bare metal

(EV) Toshy env module sees this environment:
		 DISTRO_NAME     = 'ubuntu'
		 DISTRO_VER      = '23.04'
		 SESSION_TYPE    = 'x11'
		 DESKTOP_ENV     = 'gnome'

Problem observed:


Basically everything seems to do the wrong thing now :(

For a simple example:

If i press command + X (in firefox, while writing this) I see a jumble of characters which eventually resolves to
Then command + V and i see the same jumble which resolves to

I see similar behaviour in other applications too. This happens under both X11 and Wayland.

Everything was working so well! until i plugged a new monitor in and set it all up, then went back to work and it was like this. I tried downloading the latest version and re-running the setup script to no avail.

Here is the output from toshy-config-start-verbose while selecting some text in this issue template and trying to cut/paste it using command key.

It seems to be outputting all these extra key presses. Hopefully you can make some sense of it.

Stopping Toshy systemd services...

Toshy systemd services stopped.

keyszer v0.6.92
(DD) CONFIG: /home/thayward/.config/toshy/toshy_config.py
(DD) THROTTLES: Pre-key: 8ms, Post-key: 12ms

(CG) Current settings:
        calling_module      = toshy_config.py
        db_file_path        = /home/thayward/.config/toshy/toshy_user_preferences.sqlite
        gui_dark_theme      = True
        -------------------------------------------
        optspec_layout      = 'US'
        forced_numpad       = True
        media_arrows_fix    = False
        multi_lang          = False
        Caps2Cmd            = False
        Caps2Esc_Cmd        = False
        Enter2Ent_Cmd       = False
        ST3_in_VSCode       = False
        -------------------------------------------

(CG) Toshy config sees this environment:
	DISTRO_NAME      = 'ubuntu'
	DISTRO_VER       = '23.04'
	SESSION_TYPE     = 'x11'
	DESKTOP_ENV      = 'gnome'

(DD) ENVIRON: Session type: 'x11', Desktop env: 'None'
(--) WATCH: Watching for new devices to hot-plug.
(--) Autodetecting all keyboards (--device not specified)
(+K) Grabbing Logitech MX Master 3 (/dev/input/event17)
(+K) Grabbing Magic Keyboard with Numeric Keypad (/dev/input/event18)
(--) Ready to process input.

(II) in BTN_MOUSE (press)
(DD) on_key BTN_MOUSE press
(OO) press BTN_MOUSE 1686014019.7276065

(II) in BTN_MOUSE (release)
(DD) on_key BTN_MOUSE release
(OO) release BTN_MOUSE 1686014019.7789469

(II) in BTN_MOUSE (press)
(DD) on_key BTN_MOUSE press
(OO) press BTN_MOUSE 1686014023.491242

(II) in BTN_MOUSE (release)
(DD) on_key BTN_MOUSE release
(OO) release BTN_MOUSE 1686014023.5363686

(II) in BTN_MOUSE (press)
(DD) on_key BTN_MOUSE press
(OO) press BTN_MOUSE 1686014023.6613467

(II) in BTN_MOUSE (release)
(DD) on_key BTN_MOUSE release
(OO) release BTN_MOUSE 1686014023.6981978

(II) in BTN_MOUSE (press)
(DD) on_key BTN_MOUSE press
(OO) press BTN_MOUSE 1686014023.8094177

(II) in BTN_MOUSE (release)
(DD) on_key BTN_MOUSE release
(OO) release BTN_MOUSE 1686014023.837958

(II) in LEFT_META (press)
(DD) KB_TYPE: 'Magic Keyboard with Numeric Keypad' defaulting to 'Windows' type.
(DD) modmap: LEFT_META => LEFT_ALT [Cond modmap - GUI - Win kbd]
(DD) on_key LEFT_ALT press
(DD) suspending keys [LAlt<Key.LEFT_ALT>]

(II) in LEFT_META (repeat)
(DD) on_key LEFT_ALT repeat

(II) in LEFT_META (repeat)
(DD) on_key LEFT_ALT repeat

(II) in X (press)
(DD) on_key X press

(DD) WM_CLASS: 'firefox' | WM_NAME: 'New Issue · RedBearAK/toshy — Mozilla Firefox'
(DD) DEVICE: 'Magic Keyboard with Numeric Keypad' | CAPS_LOCK: 'False' | NUM_LOCK: 'False'
(DD) ACTIVE KEYMAPS:
     'OptSpecialChars - US', 'User hardware keys', 'Firefox Browsers Overrides',
     'General Web Browsers', 'Wordwise - not vscode', 'Cmd+Dot not in
      … terminals', 'GenGUI overrides: not Chromebook', 'GenGUI overrides: Ubuntu',
     'GenGUI overrides: GNOME', 'General GUI'
(DD) COMBO: LAlt-X => <function unicode_keystrokes.<locals>._unicode_keystrokes at 0x7f92b165eb60> in KMAP: 'OptSpecialChars - US'
(DD) spent modifiers [<Key.LEFT_ALT: 56>]
(DD) resuspending keys
(DD) suspending keys [LAlt<Key.LEFT_ALT>]
(DD) resuspending keys
(DD) suspending keys [LAlt<Key.LEFT_ALT>]
(OO) press LEFT_SHIFT 1686014028.226344
(OO) press LEFT_CTRL 1686014028.2263923
(OO) press U 1686014028.2345066
(OO) release U 1686014028.234539
(OO) release LEFT_CTRL 1686014028.246651
(OO) release LEFT_SHIFT 1686014028.2467225
(OO) press KEY_2 1686014028.2549772
(OO) release KEY_2 1686014028.2551134
(OO) press KEY_2 1686014028.2753158
(OO) release KEY_2 1686014028.275357
(OO) press KEY_4 1686014028.295543
(OO) release KEY_4 1686014028.2955744
(OO) press KEY_8 1686014028.3157117
(OO) release KEY_8 1686014028.315731
(OO) press ENTER 1686014028.3358645
(OO) release ENTER 1686014028.335886

(II) in X (release)
(DD) on_key X release

(II) in LEFT_META (release)
(DD) on_key LEFT_ALT release
(DD) silent lift of spent mod LEFT_ALT

(II) in ENTER (press)
(DD) on_key ENTER press
(OO) press ENTER 1686014031.1414356

(II) in ENTER (release)
(DD) on_key ENTER release
(OO) release ENTER 1686014031.142194

(II) in ENTER (press)
(DD) on_key ENTER press
(OO) press ENTER 1686014031.2607403

(II) in ENTER (release)
(DD) on_key ENTER release
(OO) release ENTER 1686014031.2788503

(II) in LEFT_META (press)
(DD) KB_TYPE: 'Magic Keyboard with Numeric Keypad' defaulting to 'Windows' type.
(DD) modmap: LEFT_META => LEFT_ALT [Cond modmap - GUI - Win kbd]
(DD) on_key LEFT_ALT press
(DD) suspending keys [LAlt<Key.LEFT_ALT>]

(II) in LEFT_META (repeat)
(DD) on_key LEFT_ALT repeat

(II) in LEFT_META (repeat)
(DD) on_key LEFT_ALT repeat

(II) in LEFT_META (repeat)
(DD) on_key LEFT_ALT repeat

(II) in LEFT_META (repeat)
(DD) on_key LEFT_ALT repeat

(II) in LEFT_META (repeat)
(DD) on_key LEFT_ALT repeat

(II) in V (press)
(DD) on_key V press

(DD) WM_CLASS: 'firefox' | WM_NAME: 'New Issue · RedBearAK/toshy — Mozilla Firefox'
(DD) DEVICE: 'Magic Keyboard with Numeric Keypad' | CAPS_LOCK: 'False' | NUM_LOCK: 'False'
(DD) ACTIVE KEYMAPS:
     'OptSpecialChars - US', 'User hardware keys', 'Firefox Browsers Overrides',
     'General Web Browsers', 'Wordwise - not vscode', 'Cmd+Dot not in
      … terminals', 'GenGUI overrides: not Chromebook', 'GenGUI overrides: Ubuntu',
     'GenGUI overrides: GNOME', 'General GUI'
(DD) COMBO: LAlt-V => <function unicode_keystrokes.<locals>._unicode_keystrokes at 0x7f92b165ede0> in KMAP: 'OptSpecialChars - US'
(DD) spent modifiers [<Key.LEFT_ALT: 56>]
(DD) resuspending keys
(DD) suspending keys [LAlt<Key.LEFT_ALT>]
(DD) resuspending keys
(DD) suspending keys [LAlt<Key.LEFT_ALT>]
(OO) press LEFT_SHIFT 1686014032.3203104
(OO) press LEFT_CTRL 1686014032.3203297
(OO) press U 1686014032.3284595
(OO) release U 1686014032.328492
(OO) release LEFT_CTRL 1686014032.340565
(OO) release LEFT_SHIFT 1686014032.3405898
(OO) press KEY_2 1686014032.348668
(OO) release KEY_2 1686014032.3486807
(OO) press KEY_2 1686014032.368812
(OO) release KEY_2 1686014032.3688288
(OO) press KEY_1 1686014032.3889627
(OO) release KEY_1 1686014032.3889852
(OO) press A 1686014032.409135
(OO) release A 1686014032.4091616
(OO) press ENTER 1686014032.4292943
(OO) release ENTER 1686014032.429315

(II) in V (release)
(DD) on_key V release

(II) in LEFT_META (release)
(DD) on_key LEFT_ALT release
(DD) silent lift of spent mod LEFT_ALT

(II) in BTN_MOUSE (press)
(DD) on_key BTN_MOUSE press
(OO) press BTN_MOUSE 1686014036.213251

(II) in BTN_MOUSE (release)
(DD) on_key BTN_MOUSE release
(OO) release BTN_MOUSE 1686014036.2548654

(II) in LEFT_CTRL (press)
(DD) KB_TYPE: 'Magic Keyboard with Numeric Keypad' defaulting to 'Windows' type.
(DD) modmap: LEFT_CTRL => LEFT_CTRL [Cond modmap - Terms - Win kbd]
(DD) on_key LEFT_CTRL press
(DD) suspending keys [LCtrl<Key.LEFT_CTRL>]

(II) in C (press)
(DD) on_key C press
(DD) resuming keys: [<Key.LEFT_CTRL: 29>]
(OO) press LEFT_CTRL 1686014037.396937
(OO) press C 1686014037.396961
^Csignal INT received
(OO) release LEFT_CTRL 1686014037.3985279
(OO) release C 1686014037.3985467
(-K) Ungrabbing: Logitech MX Master 3 (removed)
@RedBearAK
Copy link
Owner

@thtas

The "Option-key special characters" are insulted by you calling them "strange". 😆

So, there's two problems here, and I'm not sure why one of them is happening (keyboard being treated as wrong type). That's particularly odd if it was working fine before. It shouldn't have been.

I just got done updating some FAQ entries about the keyboard. But if you can find this list in your config file and make it look like this, and restart Toshy, that should clear this up. If it works I'll put it in the default config.

keyboards_Apple = [
    # Add specific Apple/Mac keyboard device names to this list
    'Mitsumi Electric Apple Extended USB Keyboard',
    'Magic Keyboard with Numeric Keypad',
    'Magic Keyboard.*',
]

Oh, and you can disable the Option-key special character scheme from the tray icon menu, or the preferences app. But it can be pretty handy, when the keyboard is being treated as the correct type.

@RedBearAK
Copy link
Owner

RedBearAK commented Jun 6, 2023

Until the keyboard lists really get populated, this is going to be one of the primary issues people will immediately run into when trying to start using Toshy. I'm targeting better ways of temporarily overcoming the issue. Right now you really have to edit the config file to get that particular device to be identified as what it should be. But all the users with "PC" keyboards will be fine, since that's the default.

Sorry about the trouble! 🙏🏽 🤔

@thtas
Copy link
Author

thtas commented Jun 6, 2023

Thanks! I tried your suggestion but sadly it didn't work. However, i did figure out a solution.

When I plugged in this new screen I thought I would use the USB hub that it has built in. I though it was working fine but actually even though the keyboard was plugged in it had reverted to using Bluetooth!.

When I plugged it back in to the original port on the motherboard, it all started working again.

There is a difference in the way the keyboard is reported:

Via bluetooth it's known as Magic Keyboard with Numeric Keypad
Via USB it's known as Apple Inc. Magic Keyboard with Numeric Keypad

Maybe something to do with it.

@RedBearAK
Copy link
Owner

Via bluetooth it's known as Magic Keyboard with Numeric Keypad
Via USB it's known as Apple Inc. Magic Keyboard with Numeric Keypad

Ah, that is interesting. But if it is still something that keyszer is seeing and grabbing as a keyboard device, the new update to the default config should match the device name and cause it to be treated as an "Apple" type from now on. You did restart or stop/restart the Toshy services after the config file change? Or quit the verbose command and run it again?

@RedBearAK RedBearAK changed the title [BUG] Strange characters inserterd when tryint to cut/paste with command + x/c [BUG] Strange characters inserted when trying to cut/paste with command + x/c Jun 6, 2023
@thtas
Copy link
Author

thtas commented Jun 6, 2023

Yeah the rules look good, but nope i just tried again (using latest main) and if pull the plug on the keyboard and revert to BT the behavior returns. Same result if I stop/start from the tray icon or from the command line via toshy-config-start-verbose

My config in ~/.config/toshy/toshy_config.py

keyboards_Apple = [
    # Add specific Apple/Mac keyboard device names to this list
    'Mitsumi Electric Apple Extended USB Keyboard',
    'Magic Keyboard with Numeric Keypad',
    'Magic Keyboard.*',
]

I would think this pattern would miss "Apple Inc. Magic Keyboard with Numeric Keypad" no? which is odd considering that's the one that /does/ work.

Is downloading the zip (or git cloning) and running the setup script the correct process for "updating" things?

@thtas
Copy link
Author

thtas commented Jun 6, 2023

I wonder if this to do with the order in which the inputs are registered.
running xinput list
With BT:

⎡ Virtual core pointer                    	id=2	[master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer              	id=4	[slave  pointer  (2)]
⎜   ↳ Logitech MX Master 3                    	id=9	[slave  pointer  (2)]
⎜   ↳ Keyszer (virtual) Keyboard              	id=14	[slave  pointer  (2)]
⎣ Virtual core keyboard                   	id=3	[master keyboard (2)]
    ↳ Virtual core XTEST keyboard             	id=5	[slave  keyboard (3)]
    ↳ Power Button                            	id=6	[slave  keyboard (3)]
    ↳ Power Button                            	id=7	[slave  keyboard (3)]
    ↳ Sleep Button                            	id=8	[slave  keyboard (3)]
    ↳ Asus WMI hotkeys                        	id=11	[slave  keyboard (3)]
    ↳ Eee PC WMI hotkeys                      	id=12	[slave  keyboard (3)]
    ↳ Logitech MX Master 3                    	id=13	[slave  keyboard (3)]
    ↳ Magic Keyboard with Numeric Keypad      	id=10	[slave  keyboard (3)]
    ↳ Keyszer (virtual) Keyboard              	id=15	[slave  keyboard (3)]

Without BT

⎡ Virtual core pointer                    	id=2	[master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer              	id=4	[slave  pointer  (2)]
⎜   ↳ Logitech MX Master 3                    	id=9	[slave  pointer  (2)]
⎜   ↳ Keyszer (virtual) Keyboard              	id=14	[slave  pointer  (2)]
⎣ Virtual core keyboard                   	id=3	[master keyboard (2)]
    ↳ Virtual core XTEST keyboard             	id=5	[slave  keyboard (3)]
    ↳ Power Button                            	id=6	[slave  keyboard (3)]
    ↳ Power Button                            	id=7	[slave  keyboard (3)]
    ↳ Sleep Button                            	id=8	[slave  keyboard (3)]
    ↳ Asus WMI hotkeys                        	id=11	[slave  keyboard (3)]
    ↳ Eee PC WMI hotkeys                      	id=12	[slave  keyboard (3)]
    ↳ Logitech MX Master 3                    	id=13	[slave  keyboard (3)]
    ↳ Keyszer (virtual) Keyboard              	id=15	[slave  keyboard (3)]
    ↳ Apple Inc. Magic Keyboard with Numeric Keypad	id=16	[slave  keyboard (3)]

@RedBearAK
Copy link
Owner

Well, there are some rather simplistic tests like looking for the string of the "type" in the actual name. With many real Apple keyboards this will simply find "Apple" in the device name and be done with it. So it's the different name when it's connected via Bluetooth that kept that from working initially. But with the new pattern it really should.

Is downloading the zip (or git cloning) and running the setup script the correct process for "updating" things?

You have to be careful with cloning unless you really wipe out the cloned folder each time or use exactly the right git commands, or the result can be very difficult to understand, and unexpected.

In my testing I tend to go to the branch I want to test and refresh the page before downloading the zip file, and I try to make sure I delete the old zip and extracted folder and EMPTY THE TRASH because I've had weird things happen in the terminal if I was in the installer folder when I trashed the older version. It would actually continue using the version of the files IN THE TRASH.

I don't think I've ever had occasion to look at xinput list and I've been messing with these keymappers for about 3 years at this point. Pretty sure what matters is that they are all separate devices. But I could be wrong.

What do you get as output from toshy-devices (just a wrapper for keyszer --list-devices but you'd have to initiate the Python virtual environment with source toshy-venv if you wanted to do that)? Should basically show the same thing, but as keyszer sees it.

@RedBearAK
Copy link
Owner

If you've done a recent reinstall from a fresh zip, you should have this Python dictionary (a set of keys:values). Try putting the device name in the dict like this:

keyboards_UserCustom_dct = {
    # Add your keyboard device here if its type is misidentified by isKBtype() function
    # Valid types to map device to: Apple, Windows, IBM, Chromebook
    # Example:
    'My Keyboard Device Name': 'Apple',
    'Magic Keyboard with Numeric Keypad': 'Apple',
}

This is checked by a different part of the function that tries to match the keyboard type. Maybe I made a mistake elsewhere in the function and this one will work better.

@RedBearAK
Copy link
Owner

@thtas

If you have the time, I would like you to try to use this version of the list where I've used wildcards in place of all "whitespace" characters (which may or may not be normal "space" characters in the device name string) and also at the beginning and end of the string. I am wondering if there aren't some invisible non-printing characters that were inadvertently included in the string when it is connected via Bluetooth. Although I'm not using ^$ characters in the pattern so that shouldn't have made a difference. 🤷🏽‍♂️

keyboards_Apple = [
    # Add specific Apple/Mac keyboard device names to this list
    'Mitsumi Electric Apple Extended USB Keyboard',
    '.*Magic.*Keyboard.*with.*Numeric.*Keypad.*',
    '.*Magic.*Keyboard.*',
]

The keyboard does show up in the keymapper log messages as being grabbed, so there really is no good reason that I know of why this should not be working both with USB and Bluetooth.

@thtas
Copy link
Author

thtas commented Jun 6, 2023

output from toshy-devices:

With usb direct

... (more unrelated above this)...
/dev/input/event18   Logitech MX Master 3                usb-0000:00:14.0-10/input2:1
/dev/input/event21   Apple Inc. Magic Keyboard with Numeric Keypad
                                                         usb-0000:00:14.0-9/input0
/dev/input/event22   Apple Inc. Magic Keyboard with Numeric Keypad
                                                         usb-0000:00:14.0-9/input1
/dev/input/event19   Keyszer (virtual) Keyboard          py-evdev-uinput

With BT

/dev/input/event18   Logitech MX Master 3                usb-0000:00:14.0-10/input2:1
/dev/input/event19   Keyszer (virtual) Keyboard          py-evdev-uinput
/dev/input/event20   Magic Keyboard with Numeric Keypad  50:84:92:9e:02:7c

@thtas
Copy link
Author

thtas commented Jun 6, 2023

I've tried your suggested changes to the config (after starting fresh with a copy of default config from the latest main zip) but still no changes unfortunately.

@RedBearAK
Copy link
Owner

I literally just merged some changes to the keyboard lists and the function that checks the type matching into main, so you probably won't have those changes unless you refresh and download the latest zip and run the installer again. (Making sure to wipe out what you've downloaded previously, and cd out of the previous folder in the terminal.)

There is an additional check, now happening prior to the check for the keyword being in the device name, specifically when the type is 'Apple' to try and look for 'magic' and 'keyboard' in the device name. At this point I don't really expect it to work any better than the pattern matching from the list, but you never know.

Still have no idea why this isn't matching. May have to implement a simple way to force Toshy into a specific keyboard type mode. Which would be kind of contrary to the design goals.

I'd like to see a bit of logging with the new version of the function for both the USB and Bluetooth connections.

@thtas
Copy link
Author

thtas commented Jun 6, 2023

Curiously, i also tried making keyboards_Apple just an empty array - same result!
I am editing the right config right? ~/.config/toshy/toshy_config.py Then doing a restart via the app try icon

@thtas
Copy link
Author

thtas commented Jun 6, 2023

Success! I tried your latest main code, re-ran setup and now it works when connecting via both Bluetooth and USB.

Thanks you for your help.

@RedBearAK
Copy link
Owner

Finally! 😮‍💨

Now I'm really curious about which condition in the function is getting triggered, when it is connected via USB, and when it is Bluetooth. There are different strings that come out in the logging from the different conditions.

@RedBearAK
Copy link
Owner

The function dealing with matching the keyboard type has been updated in main. In particular the functioning of the custom list has been fixed. This may identify the Magic Keyboards better even without the "hack" I had to put in earlier.

The function has worked with a Bluetooth keyboard I have (some cheap generic portable keyboard) but if anyone else can test it after a new install or re-install that would be helpful.

@RedBearAK RedBearAK pinned this issue Jun 14, 2023
@RedBearAK RedBearAK changed the title [BUG] Strange characters inserted when trying to cut/paste with command + x/c [BUG] Strange characters inserted when trying to cut/paste Jun 14, 2023
@RedBearAK
Copy link
Owner

Seems like this was resolved. If you have further problems open a new issue.

@RedBearAK
Copy link
Owner

The Option-key special characters layout has now been disabled by default when you install Toshy, for a few reasons.

One is a slight performance bump from having it disabled, which can make a difference on very old CPUs.

Another is that it mostly just gets in the way of international keyboard users, or anyone who chooses a "Macintosh" keyboard layout variant, where they'll have practically the same selection of special characters, but need to use Alt_Gr to get to them.

A third good reason is that it only works for those used to the US or ABC Extended (still US) keyboard layouts on macOS.

And lastly, it requires ibus, or something that functions like ibus and accepts the Shift+Ctrl+U shortcut to initiate Unicode character entry mode. Otherwise it just spews the Unicode address into your app and hits the Enter key.

So in a lot of situations it really doesn't work all that well. If you want it enabled you'll have to choose either the US or ABC Extended layout options in the tray icon menu, or the Toshy Preferences app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants