OSX: Treat ctrl-left-click as right click

[haldean]

All native Mac applications treat ctrl + left-click as a right-click event; Apple says in their user help guide:

On Mac computers, right click is known as secondary click or Control click. If your mouse, trackpad, or other input device doesn't include a right-click button or other way to perform a right click, just hold down the Control key on your keyboard while you click.

Since ImGui is usually fed input state by directly querying mouse state, it just sees ctrl and left-click without synthesizing them into a right click. This PR integrates the native MacOS behavior of turning ctrl+left-click into right-click when the IO config flag ConfigMacOSXBehaviors is set to true.

If this is the wrong place to do this let me know; it occurred to me that this might better belong in the examples code, but there are many frontends that support MacOS, and solving this in a central location makes sense to me.

Related to https://github.com/ocornut/imgui/issues/1765

[haldean]

Build failures seem to be related to the Metal example, which I haven't touched; it looks like master is failing for the same reason.

[ocornut]

Hello,

This is problematic because we use CTRL+Click for other operations in dear imgui (e.g. turning a slider/drag into an input box), so the solution needs to be designed while being considerate of a little more things.

(Additionally, the PR doesn't follow the coding style of the rest of the codebase)

[haldean]

My understanding was that ConfigMacOSXBehaviors changed all of the ctrl modifiers to cmd modifiers, so that editing was done using cmd+click, for example. Is that incorrect? If it is: is this the sort of thing that would be appropriate to add a new config flag for, something like ConfigCtrlClickIsRightClick?

Either way: happy to modify the coding style! I'll take a look and fix what I got wrong.

[haldean]

I believe I've got the coding style right this time; sorry to waste your time not getting that right on the first push.

[ocornut]

Thanks for revising the PR syntax :)

My understanding was that ConfigMacOSXBehaviors changed all of the ctrl modifiers to cmd modifiers, so that editing was done using cmd+click, for example. Is that incorrect?

That's incorrect indeed, it currently only affects the InputText() behavior. Perhaps the right solution is as simple as just doing this (changing more/all CTRL checks to CMD?), but it's hard to settle on a solution without a thoroughly informed research. I personally don't know OS X enough to presently do it, and would more likely take such change given a more detailed report showing that all use cases have been carefully thought of. Right now even if the PR is possibly the correct move it seems like it is coming from a slightly wreckless / ill-informed point of view so it seems like a dangerous merge. Does that sounds overly cautious?

How would such chance affect user polling io.KeyCtrl flag? (Should helpers be written for the purpose of widget code?)

[haldean]

Doesn't seem overly cautious at all! I can do a more thorough evaluation of where Cmd/Ctrl are used in imgui and collect some stuff on MacOS mouse handling guidelines and write something up.

[SheridanR]

Hi there, can this patch receive some further attention please?

As a long-time mac user I can confirm that Cmd replaces Ctrl for all the familiar keystrokes (ctrl+c, ctrl+v, and so on). In fact, Ctrl is used for very little on macs besides so-called "Control-clicking" and a few bespoke or historic keystrokes (Ctrl+C in a terminal window still cancels a process, for instance - which is incidentally great, because you can still use Cmd+C and Cmd+V in the terminal as well!)

FYI, for those unfamiliar with the "priority" of apple mod keys, it generally goes: Cmd -> Option -> Ctrl. There is no "super" key (although some APIs remap "super" to one of these). Cmd is used as the basic modifier for nearly all keystrokes. Option key is typically used as an extra modifier in a very similar way as the "Alt" key (so, programming APIs that reference "Alt" usually remap that to "Option" on macOS, which generally works great). Although "Ctrl" exists on Apple keyboards and is often mapped 1:1 with the windows "Ctrl" key in programming APIs, it is actually the least used modifier key by a wide margin. In fact I'd say it has more in common with the "super" key than any other modifier, though it is rarely associated with any OS-specific keystrokes, either.

So, if ImGui actually replaced all use of "Ctrl" as an action modifier with "Cmd" on Apple devices, it would be well in keeping with existing mac design paradigms, and I really doubt there would be much if any conflict (our game "Barony" does this, and it works great for us). The "Ctrl" key could then be freed for use with Ctrl+Clicking.

[ocornut]

Thanks for your insight.

Cmd is used as the basic modifier for nearly all keystrokes.

What isn't included in that "nearly all" set ?

and I really doubt there would be much if any conflict [...]The "Ctrl" key could then be freed for use with Ctrl+Clicking.

You are suggesting a bolder and more complete change than what is spelled out by the PR code (but implied in the discussion). I think this may be the right direction and answers the first question I had (and would allow to ditch/obsolete ImGuiMod_Shortcut). But before we consider automatically remapping Cmd to Ctrl automatically and then Mac-Ctrl+Click to RIght-click, I would like to have a slightly more thorough survey of uses of Mac-Ctrl in OSX software.

In the proposed scheme it would also mean that codebase using imgui would always write things down as, e.g. "Ctrl+S" but they would be effectively be treated as "Cmd+S" and potentially swapped during text display too.

Thanks!

[SheridanR]

Cmd is used as the basic modifier for nearly all keystrokes.

What isn't included in that "nearly all" set ?

By that I meant, on a mac, for any given app, you are typically hard-pressed to find keystrokes that do not use Cmd. Much like Ctrl on Windows is used in pretty much every app-specific keystroke, Cmd takes over that role 100%.

For a specific example, there is an app on macOS called "Cheat Sheet" which displays a window showing all of the keystrokes supported by any application. This window appears whenever you hold the Cmd key down for ~5 seconds. Here is what Cheat Sheet displays for Safari, the default macOS browser. Check the bottom of the last column on the right for a legend of what each symbol means.

Notice that practically every single keystroke involves Cmd. That arrangement is not unique to Safari. The paradigm holds up across the entire OS, including the desktop, the built-in apps, and all third-party apps that I have used. I cannot stress this enough: Ctrl does not have first-priority as a mod key on macOS. While it bears the same name as the Windows Ctrl key, you are actually better off thinking of it as a separate key entirely, as it behaves completely differently to the Ctrl key on that platform.

If I can amend one aspect of my last post, I would say that, as I play with the Ctrl key on my mac keyboard right now, I'm noticing it has more in common behaviorally with the "super" or "windows" key than I first gave it credit for. For instance, holding Ctrl and tapping an arrow key switches workspaces (aka desktops - macOS supports multiple desktops, like GNOME, xfce, and KDE). Tapping other keys in combination with Ctrl also activates other esoteric desktop functions. It's not clear to me what all of these are, but they generally supersede priority of the app (and as I now recall, I have accidentally switched workspaces more than once when using Ctrl + Arrow keys in third-party apps that are not aware of this design...)

With all of this in mind, if I were to advise a direction for the design of ImGui's keyboard modifier API, I would be more agnostic about the use of key names like "Ctrl" and "Alt", and instead perhaps use terms like "Mod1" and "Mod2" which are simply remapped depending on the OS. If that kind of nomenclature change is too much trouble, simply remapping "Ctrl" and "Alt" to "Cmd" and "Option" on macOS would be very sufficient (since let's be honest, maybe 10% of imgui users operate a mac at best, and the programming API certainly doesn't need to be "mac conscious" to be fully functional).

Lastly, just in case anyone isn't already aware: Cmd+C and Cmd+V already work correctly for copying & pasting in ImGui on macOS. The unusual part is having to use Ctrl to access bespoke ImGui functions, like Ctrl+Clicking an InputText to "select all."

[ocornut]

Now that I have a Macbook I would like to carry on and work on that. I posted a few things in #4084 as well. Thanks @SheridanR for your detailed update.

The PR code itself will need a rewrite as nowadays we have io.AddEventXXX api but please don't worry about this part, as I can rewrite it, the difficulty is coming up with a general design.

The general steps would be:

• [x] Completely remap CMD to io.KeyCtrl/ImGuiMod_Ctrl, and CTRL to io.KeySuper/ImGuiMod_Super. This what gets us the closest to good behavior by default, it's only slightly confusing because of the NAME of the programming fields but it seems to make the most sense and we can ample comment on it. In the ideal world we'd rename the field but it seems not legacy reasonable.
• [x] Have a pass at "undoing" some of our Mac specific remap, and generally fix others uses (some listed in #4084).
• [x] CTRL is now mostly free and we can apply the CTRL+L Click = R Click transform initially pushed by OP.
• [x] With all the above, there's no need for ImGuiMod_Shortcut (which was a dynamic redirect to Ctrl or Cmd) to exists. It can be kept as a legacy value remapped to Ctrl.
• [ ] Not strictly required but desirable: MenuItem() functions taking a string shortcut could transform the display of e.g. "Ctrl+S" into "Cmd+S". As future similar functions are more likely to take a ImGuiKeyChord this won't be a problem for them.
• [ ] Not strictly required but desirable: (Extra) for display as we later wire a system to hook/remap symbols, we could use allow user to display ⌘ (U+2318), ⌥ (U+2325), ⇧(U+21E7).

Once this is neatly done I will be able to follow by move Shortcut(), SetNextItemShortcut() apis to public api. (Note that #456 had similar but much older discussions on similar topic, from a time where much less was in place in dear imgui codebase, so there are useful tidbits, but it's mostly out of date - not entirely though it is technically still an open topic :).

[ocornut]

Pushed a bunch of OSX related changes in 77471066, effectively applying the Cmd->Ctrl, Ctrl->Super swap discussed above, and fixing/standardized various related things.

The Ctrl+Left Click feature is NOT yet merged, I will need to rework it and validate logic.

[ocornut]

@haldean do you remember enough to clarify this code path:

// On all frames after the first, we let control go through to allow for ctrl+right click

If I am not wrong, this logic would essentially only allow to:

• Hold Ctrl, press Left button
• While Left button is still held, press Right button
• At this point app can read the Ctrl flag (which is ImGuiMod_Super now on Mac) normally it wants to.

It seems a bit bizarre to be doing this?

Pressing Ctrl+Right button wouldn't be affected by this code path, nor needed, as it can only be reached while a Ctrl+Left click is being held.

[ocornut]

I believe I have now solved this 25e279ee It's a rework/reinterpretation of @haldean logic as underlying IO system has changed quite a bit, and previous logic could not work for various reasons. And also added automated test: https://github.com/ocornut/imgui_test_engine/commit/1fcf1d9fc4791c6fb4c1f8f4cf5a712b57b30540

Some subtleties that are "undesigned" but may be altered if we find a need for it:

• Ctrl (which on mac is ImGuiMod_Super) is always forwarded to app. I don't believe it is meaningful to not forward it (original PR hid it for one frame, then kept it visible for subsequent frames as per the comment I asked a question about previously).
• Releasing Ctrl while the button is still held keeps the forwarding in place. An alternative would be to release the button, which I find odd.

Thanks for your patience!

[cfillion]

Could this new behavior be controled separately from ConfigMacOSXBehaviors? (Or perhaps ConfigMacOSXBehaviors could accept behavior flags instead of a bool?)

I use dear imgui in an application that has a user-facing option to control this and would like to continue honoring it.

https://github.com/cfillion/reaimgui/blob/c494c79eb01eef0220634190a629b7e338e2e297/src/context.cpp#L432-L450 https://github.com/cfillion/reaimgui/blob/c494c79eb01eef0220634190a629b7e338e2e297/src/context.cpp#L396-L405

EDIT: Mhh looking at the implementation I guess I could just temporarily unset ConfigMacOSXBehaviors before doing AddMouseButtonEvent...

[ocornut]

Since this is rather standard in OSX it seems odd to want to disable it? Please see how it goes and if you find you still want to provide the option over time I don’t have any fine behaviors flags if deemed useful.

[cfillion]

Nothing I can do about the option itself: the application is not mine (I only provide a dear imgui backend&bindings as a plugin for use in other plugins and user scripts). That ctrl+click setting is even disabled by default, which I agree is odd for macOS. However it'd be even weirder if scripts using imgui behaved differently than the rest of the app they're meant to integrate into (plus removing ctrl+left click being a breaking change...).

I'll probably just do something like this and keep the existing right-click emulation code on the backend's side:

const bool oldMacOSBehaviors { io.ConfigMacOSXBehaviors };
io.ConfigMacOSXBehaviors = false; // always disable imgui's right click emulation which forwards Ctrl unlike REAPER's
io.AddMouseButtonEvent(...);
io.ConfigMacOSXBehaviors = oldMacOSBehaviors;