XKB transformations

[wismill]

This a request of comments (RFC) about an evolution of XKB and the libxkbcommon library; see this introduction for further details.

• Abstract
• Motivation
• Rationale
• Specification
• Examples
• Backwards Compatibility
• Rejected Ideas
• Open Issues

Abstract

Add a new type of XKB configuration files, transformations, in order to simplify customization of keymaps:

• use lighter syntax and less knowledge required,
• enable modifications of keymaps that were not possible before (e.g. copy and swap keys/levels);
• better integrate XKB with the desktop environments and their key binding configurations.

Sample of the new syntax:

// Set level 3 of key Q to “@” for all layouts
set symbols[3] = '@' where key = <AD01>;

// Replace CapsLock with Control
copy <LCTL> into <CAPS>;

// Swap Right Win and PrintScreen, useful on ThinkPad laptops
permute <RWIN> <PRSC>;

Motivation

There are some use cases that are poorly supported in current XKB:

• Swap keys/level. E.g. swap CapsLock with Escape.
• Clone a key/level. E.g. use CapsLock as an additional Left Control.

The limitation is that the XKB format can only set key symbols explicitly and has no generic way to swap/copy keys/levels.

Furthermore, there is always a tension between XKB options and desktop environments (DE) shortcuts. Indeed, the set of options proposed by xkeyboard-config is limited and some features are just not supported by XKB, so DE have to deal with 2 interfaces to rebind keys: XKB and their own shortcut handling. DE could produce their own XKB files, but the current API does not give access to some key properties, such as: key type, actions, modifier map, etc. This is a deliberate design choice. Besides, this would require a deep understanding of XKB and creating a XKB keymap generator, which should not be necessary for simple tasks such as “swap two keys”.

Finally, keymap components, include mechanism and rules are not always easy to grasp. It can be easier to modify a keymap once its KcCGST components are assembled, i.e. without dealing directly with KcCGST components dependencies.

Rationale

We propose to add a new kind of configuration files, transformations, in order to enable the following modifications of KcCGST components:

1. Set key properties (core properties and layout-specific).
2. Copy a key into another. It can be a complete clone (only keycode is unchanged) or only a partial copy: key core properties, layouts-specific (type, symbols, actions).
3. Copy a level into another, for a given layout. It can be a complete clone (all keys defining this level) or only a partial copy for a given set of keycodes.
4. Permute a list of keys. As for 2), it can permute whole keys or only specific key components.
5. Permute a list of levels. As for 3), it can permute levels for all keys or only a given subset.

“Transformations” are not a new keymap component, but rather just a new keymap section (syntax only). They contribute to the keymap compilation but are not encoded in the compiled keymap.

Rules are extended with a new target component (RHS): transforms.

Specification

Introduction of the xkb_transforms keymap section

xkb_keymap is extended with the xkb_transforms section.

The corresponding files are located in the directory transforms of the XKB file hierarchy.

Grammar

(partial)

- FileType ::= XKB_KEYCODES | XKB_TYPES | XKB_COMPAT | XKB_SYMBOLS | XKB_GEOMETRY
+ FileType ::= XKB_KEYCODES | XKB_TYPES | XKB_COMPAT | XKB_SYMBOLS | XKB_TRANSFORMS | XKB_GEOMETRY
  Decl ::= …
+        | Transformation
+ Transformation ::= (SET | REPLACE) TransformVarDeclList WHERE TransformSetFilters SEMI
+                  | COPY [KEY] Keyname INTO Keyname [WHERE TransformKeyFilters SEMI]
+                  | COPY [LEVEL] Levels INTO Levels [WHERE TransformLevelFilters SEMI]
+                  | PERMUTE [KEY] TwoOrMoreKeynames [WHERE TransformKeyFilters SEMI]
+                  | PERMUTE [LEVEL] TwoOrMoreLevels [WHERE TransformLevelFilters SEMI]
+ TransformVarDecl ::= TransformKeyProperty EQUALS Expr
+ TransformKeyProperty ::= KEY_TYPE | SYMBOLS
+ TransformSetFilter ::= TransformSetFilterProp EQUALS Expr

TODO: syntax & semantics

Note that contrary to the other sections, the section declaration is optional in transforms files, i.e. the following two syntaxes are equivalent. However they cannot be mixed.

// Syntax 1: explicit declaration
default xkb_transforms { BODY };

// Syntax 2: implicit declaration
BODY

Compilation stage

Keymap transformations are applied just before computing the derived keymap fields.

Edge cases

Set: level is not defined in the key type

TODO: detect auto type, else clone key type and extend it?

Extension of rules files with transforms target section

Rules grammar is updated as follow:

  File         ::= { "!" (Include | Group | RuleSet) }
  
  Include      ::= "include" <ident>
  
  Group        ::= GroupName "=" { GroupElement } "\n"
  GroupName    ::= "$"<ident>
  GroupElement ::= <ident>
  
  RuleSet      ::= Mapping { Rule }
  
  Mapping      ::= { Mlvo } "=" { Kccgst } "\n"
  Mlvo         ::= "model" | "option" | ("layout" | "variant") [ Index ]
  Index        ::= "[" ({ NumericIndex } | { SpecialIndex }) "]"
  NumericIndex ::= 1..XKB_MAX_GROUPS
  SpecialIndex ::= "single" | "first" | "later" | "any"
- Kccgst       ::= "keycodes" | "symbols" | "types" | "compat" | "geometry"
+ Kccgst       ::= "keycodes" | "symbols" | "types" | "compat" | "geometry" | "transforms"
  
  Rule         ::= { MlvoValue } "=" { KccgstValue } "\n"
  MlvoValue    ::= "*" | GroupName | <ident>
  KccgstValue  ::= <ident> [ { Qualifier } ]
  Qualifier    ::= ":" ({ NumericIndex } | "all")

Default transformations

There is a special transforms file, default, that is always included first, if present.

Examples

xkb_transforms section

RMLVO base config:

• Rules: evdev
• Model: pc105
• Layouts: us,de
• Variants: (none)
• Options: (none)

// File: xkb/transforms/default

// Set level 3 of key Q to “@” for all layouts
set symbols[3] = at where key = <AD01>; // Key type will be deduced as "FOUR_LEVEL_SEMIALPHABETIC"
set symbols[3] = '@' where key = <AD01>; // Use a Unicode character
set symbols = [any, any, at] where key = <AD01>; // `any` leaves previous symbols unchanged

// Add multiple levels to the first layout
set type="FOUR_LEVEL_PLUS_LOCK", symbols[3,4,5]=[ssharp, '§', 'ẞ'] where key = <AC02>, layout=1;
set type="FOUR_LEVEL_PLUS_LOCK", symbols=[any, any, ssharp, '§', 'ẞ'] where key = <AC02>, layout=1; // `any` leaves previous symbols unchanged

// Replace CapsLock with Control
copy <LCTL> into <CAPS>; // clone
replace symbols=[Control_Left], modmap=Control where key = <CAPS>; // explicit; reset key to given values

// Copy level 2 keypad key into its level 1, such that NumLock has no impact anymore
copy level 2 into 1 where key in [<KP1>, <KP2>, …];

// Permutation: Right Win → PrintScreen → Insert, useful on ThinkPad laptops
permute <RWIN> <PRSC> <INS>;

Rules

// file: xkb/rules/evdev

// import default evdev file
! include %S/evdev

option    = transforms
// xkb/transforms/my_transforms_file contains the section “my_section”
my_option = +my_transforms_file(my_section)

<!-- file: xkb/rules/evdev -->
…
  <optionList>
      …
      <option>
        <configItem>
          <name>my_option</name>
          <description>My option</description>
        </configItem>
      </option>
      …
  </optionList>
…

Backwards Compatibility

• Keymap compilation: This change is backward-incompatible if using the new xkb_transforms keymap section or the new transforms rules target section.
• Other: not affected, as transformations are not a keymap component. This mean in particular that keymap serialization and keyboard state are unchanged.

In order to maintain compatibility in XKB files, one may extend the system configuration by adding an extra include path and proceed in the same way than user-configuration, e.g. with evdev:

• EXTRA/XKB/PATH/rules/evdev: starts with !include evdev, then adds rules with = transforms.
• EXTRA/XKB/PATH/rules/evdev.xml: adds options that require transformations files.

Rejected Ideas

• Create A dedicated API: currently xkb_keymap is read-only and we would like to preserve that property. We could still clone the keymap and modify it within an internal API, but things get complicated once reaching the derived field step.
• Modify current keymap components syntax: since we enforce only one group per symbols section, we could not make transformation across groups in such file. It seems also clearer to have dedicated section type for this RFC’s use cases.

Open Issues

• Allow to specify levels by modifier combo rather than by index. Pros: human friendly, more reliable. Cons: more verbose, require extra processing.
• Allow to specify target layout by name.
• Allow to declare variables, in order to name indexes (levels, layouts) for clarity and change them easily (define once, change everywhere).

@bluetech @whot

[wismill]

CC @mahkoh as an XKB expert

[mahkoh]

I would not call myself an XKB expert. I just know what I know.

I wonder if it would be better to devise a completely new language that is a bit more imperative. I understand that the original declarative design of XKB might have been useful in X where it was possible to replace a single part of the keymap and then have the X server re-calculate the derived properties. (E.g. you change the modmap of a key which in turn changes which interprets are applied which in turn changes the implicit actions of the key.)

But in wayland the entire keymap is an atomic entity that is only replaced as a whole.

A new language would have to support lossless conversion (within reason) from XKB and must be able to compile to a client-side XKB keymap (i.e. a map without behaviors and actions) to be compatible with the existing clients.

What I have in mind is something like this:

import * from evdev;
import swap_caps_escape;

let map = keymap();

map[A] = [ $a ];
map[LEFTSHIFT].syms = [ $Shift_L ];
map[LEFTSHIFT].actions = [ ... ];

swap_caps_escape(map);

export default map;

For such a thing to be useful it will have to have many properties:

1. Simple things must be simple.
2. Complicated things must be possible. (E.g. custom actions allowing you to have latches take effect early or late, have modifiers break latches, etc.)
3. Excellent documentation.
4. Easier to read and write than XKB.
5. Excellent support for modularity (e.g. swap_cap_escape above).
6. Buy in from the relevant parties. (Mostly compositors since the language must support export as an XKB map for backwards compatibility.)

[wismill]

Indeed we do not need the same language for the layout DB and the keymap serialization.
What you describe could be achieved with Dhall. This could be an alternative to the current language (say “classic XKB”) for the layout DB.

But I do not see a new language for xkeyboard-config anytime soon, so meanwhile we would need something to achieve the use cases of this RFC. But then perhaps Dhall can help us here also? E.g. instead of let map = keymap(); which starts the keymap from scratch we would “import” the current keymap built from the RMLVO components.

But if we can modify the keymap, then why not import the kccgst files and bypass rules completely? I.e. have Dhall (or similar language) to both replace and enhance the current XKB language for the layout DB.

[mahkoh]

What you describe could be achieved with Dhall.

I'm not sure if a general purpose language would be the best idea. We would want a good amount of syntax sugar and implicit conversions to make simple things simple. For example, above I used the $ prefix to signify keysyms without creating conflicts with other names. We would also want things like $"Ü" (or just "Ü" with implicit inference that this must mean a keysym in this context) in line with #636.

E.g. instead of let map = keymap(); which starts the keymap from scratch we would “import” the current keymap built from the RMLVO components.

I like it.

[wismill]

I'm not sure if a general purpose language would be the best idea.

We do need a DSL, it is just that your example reminds me a lot of Dhall. We could take inspiration of it, as it provides safety guarantees. So I would advocate to start with a stripped down version of Dhall (no custom data types, etc.) with relevant syntax sugars.

But in the end what we want is a custom keymap provider, so maybe we should think of it as part of a “new” API, with a default provider: the current RMLVO + rules parser and resolver. In fact the more I think of it, the more I believe we should split libxkbcommon further into e.g. libxkbcommon-db (provider, maybe merge with libxkbcommon-registry) and e.g. libxkbcommon-{client,server} (handle state and serialization).