david-vanderson
Keyboard shortcuts for menu items
Description
The last UI functionality I need for my application is support for menu shortcuts.
On Windows and Linux, these are typically implemented using the ALT key. Pressing ALT will show the keys used to access each top-level menu. Each menu item and sub-menu item typically has an accelerator / shortcut key associated with it. Using ALT and a menu key will open the menu and allow navigation using keyboard or mouse. It is not necessary to keep ALT depressed once the first menu item is accessed. Once the menu is opened, keyboard navigation can be performed using arrow keys and enter to select,
On MacOS, the shortcuts are typically displayed next to the menu items, typically operated via the CMD key. Pressing a short-cut key doesn't interact with the menu, it just performs the action. The only support MacOS appears to provide is a nice way to display the keyboard shortcuts for each menu item. e.g. Pressing CMD-F won't open the file menu.
Discussion
Examples of the different operation system menu keyboard shortcuts is below:
- MacOs
- Windows ribbon style
- Windows traditional style / Linux style
The vscode examples show the keyboard shortcuts next to each menu item. This is not always the case for Windows / Linux applications, but appears to becoming the standard. e.g. gimp uses this on Windows and Linux.
Note: in the vscode example, the underlined menu "accelerators" are not necessarily the same as the keyboard shortcuts listed on the menu item. e.g. "New Window" can be accessed with ALT-F W as well as Ctrl-Shift-N.
From the above I'd suggest we do the following, in order of importance:
- Implement the ability to short the short-cut key next to the menu item as a first step. This would give us the same support as for MacOs, and make menus look more similar to modern MacOs, Windows and Linux menus.
- Add an option to display a tool-tip for the top-level menu, showing the keyboard shortcut for that menu, similar to the Windows ribbon menus.
- Add ability to open menu when that shortcut is pressed. (either something user can pass as an init-option or something the menu implements in it's event processing)
- Add ability to select menu items based on key presses.
- Add ability to navigate menu items via keyboard arrows, enter key etc.
Not sure about the same menu item having multiple ways to access it. I suppose there is a difference between the shortcut assigned to the action and the shortcut assigned to the menu item.
Thanks for the excellent writeup! Definitely something we need, although dvui doesn't have the concept of a single application-level menu, so we'll have to figure that out.
From the above I'd suggest we do the following, in order of importance:
1. Implement the ability to short the short-cut key next to the menu item as a first step. This would give us the same support as for MacOs, and make menus look more similar to modern MacOs, Windows and Linux menus.
Sounds great.
2. Add an option to display a tool-tip for the top-level menu, showing the keyboard shortcut for that menu, similar to the Windows ribbon menus. 3. Add ability to open menu when that shortcut is pressed. (either something user can pass as an init-option or something the menu implements in it's event processing)
These both make sense - while holding ALT the menuItems could display differently (if given a menu accelerator key), and detect when ALT+accelerator is pressed - steal focus and open the submenu. This is a bit weird because it's a place where a widget is responding to key events despite not having focus. But that seems to be how it works.
Do we need a way to suppress this somehow? I'm thinking of a widget that wants to handle ALT itself. (We have some mac keybinds for ALT/option but no windows ones). Maybe we need to delay it a frame so that the menu only sees ALT events that weren't handled from the previous frame? But then ALT wouldn't always work. Maybe we do the easy thing first and see what happens, but I guess we have to exempt mac from this somehow.
4. Add ability to select menu items based on key presses.
This is activating a menuItem by accelerator right? (Like pressing ALT-F W in your example). Do the accelerators only work if you open the menu with ALT? Or do they also work anytime the menu is open?
5. Add ability to navigate menu items via keyboard arrows, enter key etc.
Does the current keyboard navigation of the menus work for this?
Not sure about the same menu item having multiple ways to access it. I suppose there is a difference between the shortcut assigned to the action and the shortcut assigned to the menu item.
Yes it is a bit weird. We can get a lot of benefit just from adding the shortcut info to the menuItems (bringing us up to mac functionality). Do you have any sense for how important the menu accelerator stuff is?
Just wanted to say I've kind of implemented this for my app, just through making a function that builds the hotkey icons or text. I've been using the icons from here, but unfortunately theres only a few icons for keys for macos, and i dont think there are any existing icons there for other OS keys that would be nice.
Maybe we could track down some icons for those keys specifically and include them in DVUI in TVG format, if they are licensed I guess? just to be able to easily draw them.
Then I think the easiest thing in my mind would DVUI offering a high level function to just pass it a keybind and it draw a the keybind similar to a label? Then you could use that label anywhere, like on main menus as shown below.
Maybe we could track down some icons for those keys specifically and include them in DVUI in TVG format, if they are licensed I guess? just to be able to easily draw them.
Then I think the easiest thing in my mind would DVUI offering a high level function to just pass it a keybind and it draw a the keybind similar to a label? Then you could use that label anywhere, like on main menus as shown below.
Yes those both sound great! Can we start from your keybindLabels() (https://github.com/foxnne/pixi/blob/30c1f2d2325d2e644f639b7d45db656bd1e3d748/src/dvui.zig#L144)?
Absolutely! I think I just copied and modified some of your code from fmt somewhere else... currently it just writes the modifier keys out i.e. ctrl, or on macos it will use the command symbol... but we need more symbol icons to really make it nice. It looks like google's material icons have key symbols, and supposedly are open source?
@foxnne That looks nice. Exactly the type of thing I was thinking of.
Thanks for the excellent writeup! Definitely something we need, although dvui doesn't have the concept of a single application-level menu, so we'll have to figure that out.
Maybe that is not needed? If each top-level menu only responds to its own short-cuts, does it matter if it is application level or not?
Do we need a way to suppress this somehow? I'm thinking of a widget that wants to handle ALT itself. (We have some mac keybinds for ALT/option but no windows ones). Maybe we need to delay it a frame so that the menu only sees ALT events that weren't handled from the previous frame? But then ALT wouldn't always work. Maybe we do the easy thing first and see what happens, but I guess we have to exempt mac from this somehow.
As far as I can tell these shortcuts are "global" i.e. ALT-F to open the file menu will work always, regardless of where the focus is within the application. And pressing alt once or holding down ALT will show the shortcuts, regardless of focus. On Mac we could change this to CMD to avoid conflicts, or disable it.
Having the option for the user to disable and implement the key handling themselves is probably a good idea. If we need the option to disable for MacOS anyway....
4. Add ability to select menu items based on key presses.This is activating a menuItem by accelerator right? (Like pressing ALT-F W in your example). Do the accelerators only work if you open the menu with ALT? Or do they also work anytime the menu is open?
Yes, this is for the accelerators. They work whenever the menu is open. Once open, the menu can be navigated by any combination of keyboard and mouse. Just pressing W when the file menu is open will operate the New Window item. (Alt-W works as well)
5. Add ability to navigate menu items via keyboard arrows, enter key etc.Does the current keyboard navigation of the menus work for this?
It sort of works. Some of the issues I found:
- If I open the File menu, but don't give focus to a menu item, I can't navigate to the menu items, but can navigate to the next menu
- If I navigate to the next menu from (1), I can then I can select the sub menus
- There doesn't seem to be a way to get from the menu items back to the menu to them navigate left-right again.
- Focus gets lost sometimes and I'm not sure how. Focus would sometimes go to the "Show Dialog Outside Frame" button in the demo when trying to navigate the menus.
- Doesn't always react to keypresses. Need more investigation to recreate consistently.
Not sure about the same menu item having multiple ways to access it. I suppose there is a difference between the shortcut assigned to the action and the shortcut assigned to the menu item.
Yes it is a bit weird. We can get a lot of benefit just from adding the shortcut info to the menuItems (bringing us up to mac functionality). Do you have any sense for how important the menu accelerator stuff is?
It's pretty standard on Windows that Alt-F N creates a new document, Alt-F X quits. Alt-E C copies etc. I would say it is generally expected that these standard menu keys work in most applications. Even the ribbon-style menus preserve the usual accelerators. And thinking about it more, it is pretty common that the keys are different. i.e. Alt-F N and Ctrl-N are both for "New", Alt-E C and Ctrl-C are both "Copy". I think most menu items will have accelerators, but only some of them will have keyboard shortcuts.
But I agree, just getting the shortcut keys in the menu is a good first step and gives a lot of value.
The keyboard navigation is probably more of an accessibility issue. I doubt that gets a lot of usage outside of that.
It sort of works. Some of the issues I found:
Yes, looks like we have some work in this area. Also see #521
- If I open the File menu, but don't give focus to a menu item, I can't navigate to the menu items, but can navigate to the next menu
Looks like opening a menu with the mouse screws up keyboard nav.
If I navigate to the next menu from (1), I can then I can select the sub menus
There doesn't seem to be a way to get from the menu items back to the menu to them navigate left-right again.
Usually we are doing "pressing left or esc closes a submenu", but this clearly isn't enough.
- Focus gets lost sometimes and I'm not sure how. Focus would sometimes go to the "Show Dialog Outside Frame" button in the demo when trying to navigate the menus.
Possibly this is due to menus doing a simple "tabIndexNext()" when you press the right arrow button, which can move focus outside the menu. This needs to be fixed.
- Doesn't always react to keypresses. Need more investigation to recreate consistently.
The menu and menuItem code definitely needs reworking. Hopefully I'll get to that soon!