FullScreen Mode Issue on MacBook Air M3: Toolbar Buttons Misaligned and Unresponsive Due to Notch

[qqiu]

Version/Branch of Dear ImGui:

Version v1.90.9, Branch: master

Back-ends:

imgui_impl_sdl3.cpp

Compiler, OS:

macOS 14.5

Full config/build information:

No response

Details:

I've encountered an issue when running an application on a MacBook Air with an M3 chip and a notch on the screen. In full screen mode, the positions of the top toolbar buttons are incorrectly calculated because of the notch. This miscalculation also causes the buttons to not respond to mouse interactions.

Screenshots/Video:

No response

Minimal, Complete and Verifiable Example code:

// Here's some code anyone can copy and paste to reproduce your issue
ImGui::Begin("Example Bug");
MoreCodeToExplainMyIssue();
ImGui::End();

[ocornut]

Please try open Tools->Metrics and report mouse inputs coordinates in a video showcasing the problem. It would also be good to see if the same issue happens with other backends.

[qqiu-cyt]

To update my earlier description: the position offset caused by the notch affects all widgets, not just the top toolbar buttons. The attached screenshot shows the cursor positioned well above the "Inputs" menu item, which is nonetheless highlighted.

[qqiu-cyt]

MacBook with the M2 chip also faces the same issue in fullscreen mode, since it has a display with a notch as well.

[coolfrik]

MacBook 14' with the M3 pro chip also faces the same issue in fullscreen mode on Main Screen. I'm looking for the problem and so far all I've found is incorrect coordinates from the macos API method addLocalMonitorForEventsMatchingMask; In this file: imgui_impl_osx.mm

...
bd->Monitor = [NSEvent addLocalMonitorForEventsMatchingMask:eventMask
                                                        handler:^NSEvent* _Nullable(NSEvent* event)
{
    ImGui_ImplOSX_HandleEvent(event, view);
    return event;
}];
...

When there are about 40-50 pixels left from the top edge, after another mouse movement, they suddenly change to +-900. I continue to search for a solution to the problem... 🤔

[coolfrik]

I found one option, it is to take CGEvent from event. It does not distort coordinates,

CGEventRef cgEvent = [event CGEvent];
if (cgEvent) {
    CGPoint screenLocation = CGEventGetLocation(cgEvent);
    NSLog(@"Window location: %@", NSStringFromPoint(windowLocation));
    NSLog(@"Screen location: (%f, %f)", screenLocation.x, screenLocation.y);
}

but there are various nuances: for example, the magic 40px that must be subtracted when you are in fullscreen mode. Although the notch size is 32px. https://developer.apple.com/documentation/AppKit/NSScreen/visibleFrame?language=objc

NSScreen *screen = [NSScreen mainScreen];
NSRect visibleFrame = screen.visibleFrame;
NSRect fullFrame = screen.frame;
NSLog(@"Visible frame: %@", NSStringFromRect(visibleFrame));
NSLog(@"Full frame: %@", NSStringFromRect(fullFrame));

In general, the above option works quite well, but with the necessary modifications.

Later i came across this documentation: https://developer.apple.com/design/human-interface-guidelines/layout where it was said:

Avoid displaying content within the camera housing at the top edge of the window. For developer guidance, see NSPrefersDisplaySafeAreaCompatibilityMode

https://developer.apple.com/documentation/bundleresources/information-property-list/nsprefersdisplaysafeareacompatibilitymode?language=objc

And magic, setting NSPrefersDisplaySafeAreaCompatibilityMode = YES, fixes everything! But there is one nuance, a black frame appears on the sides https://forums.developer.apple.com/forums/thread/693315 This is the fastest and easiest way.

Check this out @qqiu, @qqiu-cyt.

[Giometric]

I'm running into this as well on an M3 MacBook Pro, using the SDL3 + OpenGL2 implementations.

It appears to be related to this: https://github.com/libsdl-org/SDL/issues/10441

To summarize the link, when in fullscreen on a Mac with a notch, SDL_GetWindowPosition, which the ImGui implementation is relying on to get the mouse position, is not returning the values you would expect when compared to what SDL_GetGlobalMouseState is returning.

The ImGui implementation calculates the mouse position by asking SDL for the global mouse position, and comparing that to the window position, which it expects to be in the same 'coordinate space', so to speak. In this particular case, the coordinate spaces don't line up, causing the issue.

It looks like on SDL's end they don't plan to fix this since it's expected, more or less, and the correct workaround is to use SDL_GetMouseState (instead of SDL_GetGlobalMouseState). ImGui does need to be able to track the mouse outside the window when not in fullscreen, so this probably can't be used all the time, but I think checking for fullscreen mode and using SDL_GetMouseState in that case should be ok.

Below is a quick code change that appears to work on my end (the SDL_GetWindowFlags bit):

imgui/imgui_impl_sdl3.cpp:

// (Optional) Fallback to provide mouse position when focused (SDL_EVENT_MOUSE_MOTION already provides this when hovered or captured)
const bool is_relative_mouse_mode = SDL_GetWindowRelativeMouseMode(bd->Window);
if (bd->MouseCanUseGlobalState && bd->MouseButtonsDown == 0 && !is_relative_mouse_mode)
{
    // Single-viewport mode: mouse position in client window coordinates (io.MousePos is (0,0) when the mouse is on the upper-left corner of the app window)
// ----- New code here -----
    if ((SDL_GetWindowFlags(bd->Window) & SDL_WINDOW_FULLSCREEN) != 0)
    {
        float mouse_x, mouse_y;
        SDL_GetMouseState(&mouse_x, &mouse_y);
        io.AddMousePosEvent(mouse_x, mouse_y);
    }
    else
// ----- New code end -----
    {
        float mouse_x_global, mouse_y_global;
        int window_x, window_y;
        SDL_GetGlobalMouseState(&mouse_x_global, &mouse_y_global);
        SDL_GetWindowPosition(focused_window, &window_x, &window_y);
        io.AddMousePosEvent(mouse_x_global - window_x, mouse_y_global - window_y);
    }
}

EDIT: Sure enough, I should have checked the PRs first! Here's one that addresses this exact thing: https://github.com/ocornut/imgui/issues/7786

[ocornut]

Sorry for my late answer. I've posted a question at https://github.com/libsdl-org/SDL/issues/10441 because I'm not entirely sure this isn't a bug in SDL3.

[ocornut]

PS: It is entirely possible that since the issue might have been fixed in SDL3, e.g. @Giometric would be able to update to latest SDL3 and confirm if that issue is still there?

[Giometric]

Appreciate you taking a look! I double-checked using the latest release version of SDL, 3.2.22, and the issue is still present (the fix I posted previously does still work).

Based on the comments in the other thread, it does not sound like they consider this behavior a bug, but I'm not sure. At the very least it's not what I would expect, and unfortunately my version of the fix at least doesn't cover all the cases you mentioned which were the reasons for using SDL_GetGlobalMouseState to begin with.

I created some screenshots to help illustrate the issue a bit:

[ocornut]

Thanks, it's been fixed in SDL now https://github.com/libsdl-org/SDL/issues/10441#issuecomment-3326352195 But I'll push workarounds (for current version) + comments on our side.

[ocornut]

Thanks Sam! The solution on dear imgui SDL2/SDL3 backend side also becomes very simple: f61a7ef Essentially since that codepath was a workaround to provide unclamped mouse position for windows that are focused but not hovered not captured, we can simply avoid the path when SDL_GetMouseFocus() != NULL as in that situation we got SDL mouse motion events, which are both faster on X11 and correct with macOS notch with SDL <3.3.0. Thanks everyone for reporting, and sorry it took a long time to get through this.