Enhance macos keyboard api

[SergeevPavel]

• Analyse existing solutions

  • gpui (used is zed editor)
    • reimplemented charactersByApplyingModifiers wtih chars_for_modified_key
    • based on some heuristics zed decides which modifiers to apply to key NSEvent.keyCode to get key and key_char
    • for Russian and Dworak layout it sends cmd + keyCode as key which turn letters to latin
  • jwm
    • the code is quite similar to jdk
    • uses key code and ignores characters
    • https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/19fb8f93c59dfd791f62d41f332db9e306bc1422/src/java.desktop/macosx/native/libawt_lwawt/awt/AWTView.m#L282
  • keyboard-types https://github.com/pyfisch/keyboard-types
  • jetbrains runtime
    • https://github.com/JetBrains/JetBrainsRuntime/blob/aa8fe49bcd391009472c884ae88b6a0fd4e66e88/src/java.desktop/macosx/native/libawt_lwawt/awt/AWTView.m#L373
  • chromium https://source.chromium.org/chromium/chromium/src/+/main:docs/ui/input_event/index.md?q=interpretKeyEvents
  • firefox https://searchfox.org/mozilla-central/source/widget/cocoa/nsChildView.mm#3568
  • web
    • input methods for web https://developer.mozilla.org/en-US/docs/Web/API/EditContext_API/Guide
    • keyboard layout api https://developer.mozilla.org/en-US/docs/Web/API/Keyboard_API
      • allow to override browser/system shortcuts
      • allow to convert key codes to text on the key
    • keyboard event https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent
      • contains code physical key location e.g. KeyQ
      • contains key which takes into account keyboard layout, locale and pressed modifiers
        • NB for dead keys key is Dead
      • https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent

• Proposed solution:

  • apply NSApplication hack to fix keyUp events
  • override performKeyEquivalent to prevent app menu from handling shortcuts
  • on every keyDown and keyUp event:
    • retrieve virtual key code
    • retrieve key with pressed modifiers
      • decode keys like enter, esc, f1, f2, ...
      • other keys represent as a string
    • retrieve key without modifiers
  • if event wasn't handled, feed it to input method system

• API methods thats is usually used

  • NSResponder
    • keyDown: / keyUp:
      • aren't triggered for modifiers
    • NSEvent charactersByApplyingModifiers
      • allow to get layout dependent characters ignoring all modifiers
    • flagsChanged triggered when modifier pressed
    • performKeyEquivalent macos ask to trigger an action for event
      • triggered when cmd or ctrl modifier is pressed
      • gpui overrides it to go before app menu
        • zed-industries/zed@8f6d081
    • _wantsKeyDownForEvent undocumented api used by chrome and firefox
      • it seems that this method is ignored now
  • willBeHandledByComplexInputMethod undocumented method of NSEvent
  • interpretKeyEvents we send event back to NSResponder

• Problems

  • keyUp and keyDown isn't always paired => fixed by Application sendEvent patching from JWM
    collapsed:: true
  • check that we can handle Dvorak-QUERTY ⌘ https://youtrack.jetbrains.com/issue/FL-11912/Support-for-Transmorphic-Keyboard-Layouts
    • three options for handling shortcuts
      • use US-ANSI keys (NSEvent key code?) for shortcuts, that corresponds physical keyboard position [the way of JWM]
      • use layout dependent key code, ignore modifiers (NSEvent charactersIgnoringModifiers?, it's almost what we need but unfortunately it contains shift modifer)
      • use layout dependent key code with modifiers (NSEvent characters?)

[SergeevPavel]

To experiment and verify the quality we could:

• Implement a simple text input component in SkikoSample some code can be stollen from here
• Write Kotlin integration tests, that interacts with real UI by sending events like awt robot
• Headless integration tests in Kotlin (which can emulate NSApplication events)
• Unit tests in Rust