MuseScore Dropdowns, menus and lists should highlight items in reaction to key presses ("Initial Letter Navigation")

Task description When a dropdown, menu or list is focused, you should be able to type text on the keyboard, to quickly jump to the corresponding item.

For example, in the dropdown shown below,

when you press S, it should highlight the ~first~ next item that comes lexicographically after "S", so in this case "Sans Serif Collection".
when you press S E in quick succession, it should highlight the ~first~ next item that comes lexicographically after "Se", so in this case "Segoe Fluent Icons".
when you press S G in quick succession, it should highlight the ~first~ next item that comes lexicographically after "Sg", so in this case "Showcard Gothic".

Edits by @shoogle:

Changed 'first' to 'next'.
The delay for "in quick succession" is one second (see WM_CHAR), but could be made configurable in Preferences > Advanced. Within this period, letters are added to a string buffer and a search is performed for the entire string. If the current item starts with the string then selection remains on this item, otherwise it goes to the next item that starts with the string. After the period expires, the string buffer is emptied. A subsequent key press starts a fresh search with a new string buffer, and will move to the next matching item even if the current item is a match.
If the first letter is pressed repeatedly (e.g. S S S) then the selection cycles through items that begin with that letter (e.g. "Sans", "Sergeo", "Showcard"). However, the buffer is still saved as SSS, and if a different character is pressed before it empties (e.g. SSSQ) then the selection jumps to the next item starting with this entire string (e.g. "Sssquash").
See PR #5829 for previous implementation in MuseScore 3. If in doubt, check the behaviour of Windows File Explorer.

Technical note This will need to be implemented for all QML dropdowns/menus/lists/treeViews. In Qt Widgets, this already works out of the box (and for those few cases where it doesn't, we shouldn't invest time to fix it since Qt Widgets is going to be replaced eventually anyway).

Feb 23 '23 17:02 cbjeukendrup

I agree. I also think you've basically designed it, so no input required from me. Let's do it!

Feb 23 '23 17:02 Tantacrul

Can we add a feature where the font selection drop-down also shows what the font look like?

Feb 24 '23 13:02 octopols

@octopols Please create a separate issue for that

Feb 24 '23 15:02 cbjeukendrup

Really hope this one can be addressed as it is standard accessibility practice and makes a significant difference to productivity. Take for example adding the swing item from the tempo palette, currently that takes 31 down arrow presses to get to!

Nov 05 '23 00:11 jmwmusic

@jmwmusic if you can, would you avoid 'hurry up' messages, which sends us all emails? It'll get done when it gets done. The fact that it's an active issue is enough.

Nov 05 '23 02:11 Tantacrul

Sorry, was not meaning that as a hurry up, just checking it was still being looked at as the other ticket I opened got a response added that there was no plan for this, which seemed strange as it also linked back to this one, so just making sure the need for it as an accessibility issue was understood. I think I didn't find it in my initial search as already existing because it didn't have an accessibility tag possibly until now?

Nov 05 '23 02:11 jmwmusic

The "closed as not planned" email is a standard GitHub thing, used for things like duplicates as well as things that are by design; basically, any reason apart from that the issue is solved. So in the case of a duplicate, it just means that the duplicate is moved out of the way, while the original continues to exist. (Don't worry about not finding this issue; sometimes finding the right search term is not quite obvious.)

Nov 05 '23 11:11 cbjeukendrup

Can I check whether the scope of this issue currently includes palette lists, and if not can that be added?

Nov 09 '23 22:11 jmwmusic

I think this might be related - Currently screen reader is not announcing the Mnemonic Shortcut options in menus. So for example in the File menu if you arrow down to Open it just says "Open Ctrl O" but would usually say "Open Ctrl O O". But maybe this is deliberate till it is working as currently if I press Alt F O I get stuck in nomans land with no focus, tab & arrow do not work and I have to press Esc to get out of it.

Nov 10 '23 00:11 jmwmusic

I think the Mnemonics issue is unrelated, and it would be preferable to have a separate issue for that. MuseScore uses a custom menu implementation on Windows, so speech for mnemonic shortcuts was probably just not implemented. However, mnemonics themselves should be working correctly. If not, that would be a regression and would deserve an issue on its own.

Nov 10 '23 00:11 cbjeukendrup

Hey, I would like to work on it :)

Jan 16 '24 15:01 ketgg

Issue reported here on .org, along with issues with the Up arrow keystroke.

Apr 04 '24 05:04 scorster

Came up again in https://musescore.org/en/node/366117

Jul 13 '24 20:07 Jojo-Schmitz

As noted in https://github.com/musescore/MuseScore/issues/22179#issuecomment-2227518611, this is a bit worse than just "not working yet": pressing a letter key while a list is focused is interpreted as activating a global keyboard shortcut, resulting in notes unexpectedly being repitched for example.

Sep 06 '24 15:09 cbjeukendrup

Working on this.

Sep 08 '24 17:09 shubham-shinde-442

@cbjeukendrup I worked on this issue earlier. It seems like the focus is disabled for dropdowns by this line: openPolicies: PopupView.NoActivateFocus in StyledDropdownView.qml. Because of this, the code in Keys.onShortcutOverride doesn't work and instead the global keyboard shortcut is activated as ShortcutsController::activate is called on pressing any key.

@Eism mentioned that this was done intentionally so that opening dropdowns wouldn't steal focus from the current window.

Hope this helps you, @shubham-shinde-442 :)

Sep 08 '24 19:09 ketgg

@Eism mentioned that this was done intentionally so that opening dropdowns wouldn't steal focus from the current window.

Why should the score respond to keystrokes made when the user has a menu open? If the user expects/wants their keystroke to interact with the score, but notices that it doesn't, they can simple escape the menu and press the keystroke again. Seems like standard fare with lists and menus.

What am I missing here?

MuseScore exhibited lots of focus issues over the years. Thankfully many of them have been fixed, resulting in improved and mainstream UX. I hope to see this resolved. Searching font menus can be a pain.

Sep 08 '24 21:09 scorster

What am I missing here?

I'm not sure, but possible reasons might be:

To allow changing the dropdown value with Up and Down arrow keys without opening it first (but I think this only works in MS3 and not MS4 anyway).
The screen reader might treat the dropdown as a new window and say the window name when you focus/unfocus it, which could be confusing/unhelpful.
Focusing might break speech in some other way, perhaps because we use our own accessibility system rather than Qt's system.
Focusing might break keyboard navigation somehow, perhaps because we use our own shortcut-based navigation rather than Qt navigation.
If you open the Playback or Appearance popup in the Properties panel, these popups are supposed to stay open while you navigate the score. Perhaps they share some code with dropdowns, and keeping them open means focus has to stay in the main window.

@shubham-shinde-442, you could try focusing the dropdown if you like. If you do, those are the things I would watch out for.

Sep 09 '24 11:09 shoogle