Accessibility tools - Virtual Keyboard

[AdrienPiechocki]

Pull Request

Motivation

I made a virtual keyboard for accessibility purposes. I don't know if it would be useful since accessibility isn't much of a thing yet. But here it is !

**This panel requires a new package : ** ydotool

Should ydotool be a dependency ? I'm not sure since it is only for that panel to work. Plus, I have no idea on how to do so...

You can toggle the virtual keyboard with ipc call keyboard toggle or with the new bar widget (see video)

The keyboard layout is either QWERTY or AZERTY based on your actual layout (thanks to Noctalia's KeyboardLayoutService).

Click the keys to type them on where you were last focused.

Modifiers such as "shift" or "super" work as intended.

Type of Change

Mark the relevant option with an "x".

• [ ] Bug fix
• [X] New feature
• [ ] Breaking change
• [ ] Refactoring

Related Issue

None.

Testing

Describe how you tested your changes and mark the relevant items.

• [ ] Tested on niri
• [X] Tested on Hyprland
• [ ] Tested on sway
• [X] Tested with different bar positions and density settings
• [X] Tested at different interface scaling values
• [X] Tested with multiple monitors (if applicable)

Screenshots / Videos

https://github.com/user-attachments/assets/80a3d3d3-6138-4a4c-9f0c-4ad166b1fb78

Checklist

• [X] Code follows project style guidelines
• [X] Self-reviewed my code
• [X] No new warnings or errors
• [X] Documentation or comments updated (if relevant)

Additional Notes

Some improvements could be made. Such as a keyboard layout selection option (instead of the current automatic detection). I should probably make relevant keys change their text when shift is pressed.

edit : Relevant keys are now displayed when shift is pressed.

[AdrienPiechocki]

Another thing I noticed when testing is that panels "steal" focus (except the virtual keyboard because I tweaked it).

So if you have the app launcher toggle set to SUPER + R and press those keys via the virtual keyboard, the app launcher opens but "steals" focus ; meaning you have to close and reopen the virtual keyboard if you intend to use it on the app launcher.

That may not be the best example but it is something to keep in mind I guess.

[AdrienPiechocki]

Regarding my last comment, it seems you can only have one panel opened at once (which wasn't the case when I was testing at first). Which means you can't use the virtual keyboard along with other panels such as the app launcher.

[AdrienPiechocki]

I just added settings to my virtual keyboard. You can now change the keyboard's position on screen via the settings panel, under the accessibility tab. I made a French and English translation (because I speak those languages) but was wondering if it is okay to use online translation tools for the other languages ? Or maybe I should let the community build their own as needed ?

[AdrienPiechocki]

Since last push introduces quite big changes to the UI of the virtual keyboard, I figured i'd make a video to showcase it.

https://github.com/user-attachments/assets/e8bc689c-694f-4ed0-9431-f53dfd5f83dd

The virtual keyboard is now a FLOATING WIDGET ; not a panel anymore. This allows moving the keyboard where you want on screen and allows using it on SmartPanels such as the App Launcher. A few issues remain : I'd like the keyboard to be scaleable in real time via drag n' drop. The keyboard can't be dragged from a screen to another.

[ItsLemmy]

Hey Adrien, just so you know we are working (and are almost done) with a new plugin system. We feel like your PR would be great as a plugin rather than a built in widget. The conversion should be rather straightforward. We will keep you posted when we are ready.

[AdrienPiechocki]

That is great ! Thank you for letting me know :D I'll keep adding a few fixes and smaller features I have in mind then convert my work as a plugin.

[ItsLemmy]

Plugins are now a thing in the git branch. Official release will probably be on Sunday. Feel free to give it a shot.

[AdrienPiechocki]

Thank you @ItsLemmy ! I made the plugin and it works almost perfectly ! The only issue I have is the fact that SmartPanels always "steal" focus. I managed to change that but it requires a small (non-breaking) edit to the main code. I pushed what was needed in this PR (and removed the rest).

What I did is adding a new setting for a new kind of panel : floating panels. Since any PanelWindow that is NOT loaded with MainScreen.qml is always on top (let's call them floating panels), you can have them along with a SmartPanel. However, since SmartPanels "steal" focus, trying to interact with the floating panel is impossible (you will always click or type inside the SmartPanel).

So, with that last commit, I made MainScreen.qml disable its focus if a floating panel is active and gave the option to re-enable focus to MainScreen and its SmartPanels even if a floating panel is active. This is useful if the user wants to type something inside a SmartPanel while a floating panel is active.

Anyways, this last commit is necessary for my virtual keyboard plugin to work with Noctalia's Panels. Is that okay to add ? If you have questions or want to check the code of my plugin, feel free to ask !

[ItsLemmy]

Can you maybe open a PR on the plugins repo with the plugin so I can test it. Gotta be extra carefull with keyboard focus stuff, and test on all compositors.