Allow selecting keys by pressed arrow keys

[PureTryOut]

I'm thinking of being able to use Maliit for Plasma Bigscreen. Currently they use Qt5VirtualKeyboard as that allows navigating the keyboard using key navigation. They require to be able to accept input from remote (TV) controls which often only have arrow keys available, and they emulate the arrow keys from a keyboard.

[jpetersen]

Yes a good idea for an use case not 100% sure how it works with focus etc to get the input events.

But I think we should support it in general

[plata]

Having the "up" key in terminal would also be very helpful.

[dobey]

This was also opened at https://invent.kde.org/plasma/plasma-bigscreen/-/issues/16 for that side of things

[jbaker0428]

I'm down this rabbit hole because I'm working on a custom layout for myself and wanted arrow keys to use on a Surface Pro that I use as a media tablet. Now, with the caveats that I'm a hardware engineer that only rarely touches code, am totally new to QML and this codebase, and don't have a proper IDE set up... I think this might already be mostly implemented. Forgive the thinking out loud as I go searching through the repo, but I think I've possibly figured it out.

First of all, someone was thinking about this in the past. The Features page claims hardware keyboard support for:

Long pressing of keys either to autorepeat (e.g. arrow keys, backspace)

The first thing I came upon while branching out from my custom layout files is in Keyboard.qml, starting at line 444: https://github.com/maliit/keyboard/blob/master/qml/Keyboard.qml#L444

// Autopilot needs to be able to move the cursor even when the layout
// doesn't provide arrow keys (e.g. in phone mode)
    function sendLeftKey() {
        event_handler.onKeyReleased("", "left");
    }
    function sendRightKey() {
        event_handler.onKeyReleased("", "right");
    }
    function sendUpKey() {
        event_handler.onKeyReleased("", "up");
    }
    function sendDownKey() {
        event_handler.onKeyReleased("", "down");
    }
    function sendHomeKey() {
        event_handler.onKeyReleased("", "home");
    }
    function sendEndKey() {
        event_handler.onKeyReleased("", "end");
    }
    function selectLeft() {
        event_handler.onKeyReleased("SelectPreviousChar", "keysequence");
    }
    function selectRight() {
        event_handler.onKeyReleased("SelectNextChar", "keysequence");
    }
    function selectUp() {
        event_handler.onKeyReleased("SelectPreviousLine", "keysequence");
    }
    function selectDown() {
        event_handler.onKeyReleased("SelectNextLine", "keysequence");
    }

These function definitions appear to be different from how the other special keys that are in-use on keyboard layouts are defined. For example, let's look at how the Backspace key is defined here: https://github.com/maliit/keyboard/blob/master/qml/keys/BackspaceKey.qml

import QtQuick 2.4

ActionKey {
    iconNormal: "edit-clear-symoblic";
    iconShifted: "edit-clear-symbolic";
    iconCapsLock: "edit-clear-symbolic";
    action: "backspace";
}

The backspace key is then added to keyboard layout QML files with just one line: BackspaceKey {}

So I think we just need additional key files for the arrow keys, the question is just what those look like. Digging around the same folders some more, I found similar Action (not ActionKey) definitions in ActionsToolbar.qml: https://github.com/maliit/keyboard/blob/74f3b0b790c34ae60b9bdff575d3bc079af75619/qml/ActionsToolbar.qml#L35

            readonly property list<Action> actions: [
                Action { text: qsTr("Undo"); icon.name: "edit-undo-symbolic"; onTriggered: fullScreenItem.undo();},
                Action { text: qsTr("Redo"); icon.name: "edit-redo-symbolic"; onTriggered: fullScreenItem.redo();},
                Action { text: qsTr("Select All"); icon.name: "edit-select-all-symbolic"; onTriggered: fullScreenItem.selectAll(); }
            ]

I have yet to encounter this action toolbar in my own personal use, so I'm not really sure what that action list setup looks like in practice. However, importantly, those fullScreenItem calls are defined in the section of Keyboard.qml referenced above, which includes the arrow direction functions. So, one way that a hypothetical qml/keys/LeftKey.qml might work would look something like this:

import QtQuick 2.4
// maybe an extra import required? ActionsToolbar.qml imports more stuff

Action {
    text: qsTr("←");  // Is that unicode arrow character legal here?
    icon.name: "edit-left-symbolic";
    onTriggered: fullScreenItem.undo();
}

I'm not confident that this seemingly nonstandard solution is required though. That Keyboard.qml segment does not have any sort of handling for backspace, which appears to be implemented very cleanly. I started looking for where the "backspace" action comes from and found some very similar event handler functions in the logic source code proper, which includes both the backspace action as well as a full set of arrow keys and home/end: https://github.com/maliit/keyboard/blob/74f3b0b790c34ae60b9bdff575d3bc079af75619/src/lib/logic/eventhandler.cpp#L77

If this is indeed where the backspace action comes from, then our qml/keys/LeftKey.qml starts to look a lot more like Backspace.qml. I have no idea where those icon names come from or if the "left" icon graphic exists, but assuming it does:

import QtQuick 2.4

ActionKey {
    iconNormal: "edit-left-symoblic";
    iconShifted: "edit-left-symbolic";
    iconCapsLock: "edit-left-symbolic";
    action: "left";
}

Searching around for other uses of Key::ActionBackspace and Key::ActionLeft, I was able to find the matching 2013 commit to the Features page claim of arrow key repeatability: https://github.com/maliit/plugins/commit/69bfe5b0007d997fc3491c505baa1ba0244364cd

Am I missing something big under the hood? Should creating ActionKey files for the arrows not just work?

[PureTryOut]

I think you misunderstood. This is not about arrow keys on Maliit itself, but physical arrow keys on a keyboard moving key selection on Maliit. As I said in the original message for me this is mostly intended to allow navigation on the keyboard using TV remotes which emulate arrow key presses.

[jbaker0428]

Ah sorry, too far in the weeds trying to figure it out for my own uses. Wasn't sure quite what you originally meant by key navigation. So your remote sends arrow key signals, and you want it to highlight individual keys on the Maliit OSK and press an OK button or similar to type the selected letter? In that case, my ActionKey QML file part is indeed no good for you (although I may still try and make it for myself). That makes a lot more sense as to why this would be a focus concern.

Hopefully this is at least a good reference for where the existing code works with arrow keys. Assuming the focus problem can be sorted out, you would then need similar (perhaps the same?) event handlers to move your selection cursor. In the use cases I understand, those all handle an action event that's coming from Maliit itself. In this case, you need Maliit to know that you pressed the arrow key on effectively a physical keyboard. There's some other references in MeeGoo-era documentation to handling hardware keyboard events, so hopefully catching the signal in the first place wouldn't be difficult.

[PureTryOut]

So your remote sends arrow key signals, and you want it to highlight individual keys on the Maliit OSK and press an OK button or similar to type the selected letter?

Exactly.

[plata]

Maybe reword the issue title then. I also misunderstood the purpose.

See also #54 (@dobey).