Window hints

[dexterleng]

[dexterleng]

@blackketter @lougreenwood @mmroczka

Would you mind giving it a try? There are two changes:

1. Window Switching hints
2. Hints for context menus (except for those in the top-right menu icons)

vimac-window-switching.zip

[blackketter]

That's pretty neat! I like it. A LOT.

Some feedback...

Window Hints:

1. I don't think it's necessary to use different hint keys for windows. I'd rather use the existing hint key set and not have to reach for the number keys.
2. I don't think it's necessary to use a different font, shape and size for the window hints. It might make sense to use a different color for the hints, but I'd be happy if they all looked alike. It would be easier to parse, visually.
3. Sometimes windows don't have hints, I think this is when they are mostly hidden by other windows. My expectation is that if some part of a window is visible and clickable, it should have a hint.

Menu Hints:

1. There are a few spots where they are missing, but those might be application specific.
2. When I use the menu hints to pull down a menu and show a submenu, then refresh the hints, I see hints for the menubar items and hints for the submenu items, but not for the menu in between them.

I have been playing with some code that provides keys for refresh and lock of hints. These should really compliment these new menu hints!

[dexterleng]

Sometimes windows don't have hints, I think this is when they are mostly hidden by other windows. My expectation is that if some part of a window is visible and clickable, it should have a hint.

Agreed, hints are currently shown for windows whose center or top-left are visible. I'm currently using the Accessbility API's "element at position" function to test for visibility which makes it difficult/slow to test for visible areas. Ideally what I need is an algorithm to calculate the visible areas of a list of 2d rectangles with z-indices. I found Painter's algorithm which might be relevant. Although I think checking the top-right and bottom-left and bottom-right corners could solve most of these edge cases.

There are a few spots where they are missing, but those might be application specific.

Could you share examples?

When I use the menu hints to pull down a menu and show a submenu, then refresh the hints, I see hints for the menubar items and hints for the submenu items, but not for the menu in between them.

Noted.

[blackketter]

Agreed, hints are currently shown for windows whose center or top-left are visible. I'm currently using the Accessbility API's "element at position" function to test for visibility which makes it difficult/slow to test for visible areas. Ideally what I need is an algorithm to calculate the visible areas of a list of 2d rectangles with z-indices. I found Painter's algorithm which might be relevant. Although I think checking the top-right and bottom-left and bottom-right corners could solve most of these edge cases.

Sounds like a homework assignment for a CS class. :) Seems to me that you can compare two rectangles for occlusion and generate zero to 4 visible rectangles. Then recursively test those visible rectangles against the next rectangle. When you are done you have a list of visible rectangles. Exercise left to the reader.

There are a few spots where they are missing, but those might be application specific.

Could you share examples?

Looking again, the examples I see are related to menus on the right of the menubar, which you said didn't work. I do find that some applications that put up custom menus do in fact work, like iStat Menus and most of Fantastical. I'm impressed to see that widgets and notifications work too!

[blackketter]

I do see one other issue with windows. If the current frontmost application does not have any windows open, the topmost window from the other applications is not hinted.

[mmroczka]

This addition is fire! 🔥 I think the only feedback I'd have is that we shouldn't change to numbers for window hints. Reaching to the number row sucks. This mode even works well even on multiple monitors. It seems to only occasionally not recognize that there are windows on the other monitors. Generally, the same overlap problem that @blackketter mentioned. It'd be a really cool feature to make the window hints for an application always be the same and never changing. For instance, you could make a window be the first 3 letters of the application. chr for Chrome, sla for Slack, etc. You could even make this efficient by pre-processing the applications in the application folder and assigning key/value pairs for all possible open applications. Just mark those 3-letter combinations as reserved letter combinations and throw them in a set for quick-lookup during the hint generation phase.

[Signynt]

Hey, first of all, awesome app, it's really useful! As soon as I started using it I thought that it would be super useful to be able to do window switching with it, so I found this branch and tried it out, but found that it is somehow super slow.

When I open the hints the yellow clicking hints immediately show up, but I can't activate any of them. After about 10 seconds the window hints show up, after which I can now use any of the hints for the windows or for clicking. I'm on 12.1 Beta (21C5039b) on a Mac M1 Mini (2020)

Just wanted to let you know about this issue!