is_key_down keep being true when window gets unfocused

[lomstfer]

1. I press and hold down a key.
2. I make the window not in focus (on windows) and release the key.
3. I see that is_key_down of the key that I pressed is still true, until i focus the window again and release the key again.

[miwarnec]

This happens to me as well for macroquad 0.3.X. For example, on Windows: press Windows key.

• Key pressed fires once
• Key down keeps firing while in background since the start menu is now in foreground
• Switch back to macroquad application, key down still keeps firing

in case of the Windows key, this can't even be released since pressing it again opens the windows start menu again.

[profan]

Definitely also still happening on macroquad 0.4.x for me (on Ubuntu 22.04, using XFCE (Xubuntu))

Releasing keys with the mouse off the window means you don't seem to get notified of the input at all, which causes a lot of slightly funky behaviour if you happen to accidentally mouse off the window momentarily (while middle mouse panning or similar, where you want it to terminate on releasing the mouse button).

Reproduction

1. Launch the following program.
2. Press left mouse with the mouse inside the window.
3. Move the mouse out of the window, releasing left mouse after the mouse is outside of the window.
4. Observe that left mouse still appears to be down, and mouse button released never triggered.

use macroquad::prelude::*;
  
#[derive(Debug)]
pub struct ButtonState {
    pub pressed: bool,
    pub down: bool,
    pub released: bool
}

pub struct ButtonCounterState {
    pub press_count: i32,
    pub release_count: i32
}

fn get_mouse_button_state(button: MouseButton) -> ButtonState {
    ButtonState {
        pressed: is_mouse_button_pressed(button),
        down: is_mouse_button_down(button),
        released: is_mouse_button_released(button)
    }
}

#[macroquad::main("InputRelease")]
async fn main() {

    let mut mouse_left_state = ButtonState { pressed: false, down: false, released: false };
    let mut mouse_left_count = ButtonCounterState { press_count: 0, release_count: 0 };

    loop {

        clear_background(LIGHTGRAY);

        mouse_left_state = get_mouse_button_state(MouseButton::Left);
        mouse_left_count.press_count += mouse_left_state.pressed as i32;
        mouse_left_count.release_count += mouse_left_state.released as i32;

        draw_text(&format!("mouse left: {:?} (presses: {}, releases: {})", mouse_left_state, mouse_left_count.press_count, mouse_left_count.release_count), 20.0, 20.0, 16.0, DARKGRAY);

        next_frame().await

    }
}

[jangler]

Has anyone figured out a workaround for this?

[profan]

Has anyone figured out a workaround for this?

god I wish😢, if you do let me know because it's a royal pain for sure right now

[jangler]

In my use case I'm mostly concerned with predictable loss of focus from using rfd, so the workaround would be either:

1. Track input state yourself (using get_keys_pressed, etc.) and clear it when opening a file dialog
2. Patch in a method to reset the input state of macroquad's context object

But of course this is useless for scenarios where focus is lost unpredictably.

[chamons]

Looking into this, I think miniquad would need a window focus event (see note on https://docs.rs/miniquad/latest/miniquad/window/index.html) for macroquad to listen to and reset pressed keys?

[jangler]

That's my understanding!

[profan]

It looks like there's been a bandaid fix for this, but only for wayland in: https://github.com/not-fl3/miniquad/pull/519

Fairly confident this is still an issue even after that PR, at least definitely on non-linux platforms and just clearing the modifiers doesn't feel like the right fix anyways

[bolphen]

I think the keys_down and mouse_down should be cleared on the window_minimized event, which is the closest we have to a focus_lost.

    fn window_minimized_event(&mut self) {
        get_context().keys_down.clear();
        get_context().mouse_down.clear();
        #[cfg(target_os = "android")]
        get_context().audio_context.pause();
    }

At least on X11 this fixes the issue.

[jangler]

In my experience the pressed/released state also needs to be cleared.

[bolphen]

These are already being cleared every frame in Context::end_frame

[jangler]

Oh, I guess my issue was specific to blocking dialogs then!

[jonatino]

Might try and fix this soon. Is there any situation where you'd still want to accept user input when a window is not in focus? I can't think of any scenarios with touch/keyboard/mouse input where you'd want to accept input when not in focus, but what about if a controller is plugged in? Still ignore all input until window is back in focus?

[profan]

@jonatino I feel like if a key/mouse button/etc was pressed while inside the window, and you unfocus/mouse off the window, you'd still want to receive the released event for those events at least (and not just clear all the state arbitrarily when window unfocuses), at least this is the current ballache I'm having with controls being "sticky" personally

[jangler]

Clearing all the pressed keys/buttons when focus is lost is the usual behavior, though.

[jonatino]

@jangler yeah I tend to agree with you. I don't think what @profan wants is possible across most operating systems (especially on mobile/web where things tend to be a lot more sandboxed). My solution was simply to clear all input state on window defocus. The sticky key issue you have will likely be fixed by this since you'd know right away after the window loses focus that the key is no longer held (since input states are cleared).

[profan]

@jangler I don't think you'd get that on SDL by default or even just with Godot, but as long as the sticky key issue is resolved somehow I'll be placated as at least now the game will at least feel less broken :)

Or rather most importantly, what at least I need for consistency is for every input key pressed event to have a corresponding input key released event (so that things which enable on press and disable on release work properly) and not for half of the event chain to suddenly go missing.

So if your "just clearing the input state" includes closing this loop, that's totally fine, but if you're literally just clearing the state of mouse/key down states etc, it will produce a half-functioning outcome.

[jonatino]

@profan what you're asking for doesn't exist in SDL or godot and is impossible on half the platforms that macroquad supports so it will lead to inconsistent behavior in the engine which I imagine @not-fl3 doesn't want.

No operating system's window manager forwards input events to windows that aren't in focus. This is done intentionally as a security reason otherwise any app on your computer could keylog you in the background with no special permissions (since they'd be able to get input events even when not in focus). The only way to get input events out of focus would be raw input hooking but that's impossible to do on web and mobile without permissions that most people won't grant for a game.

Eg: SDL you need to specify you want to handle game controller inputs (and only controller input events) when window is unfocused. https://wiki.libsdl.org/SDL3/SDL_HINT_JOYSTICK_ALLOW_BACKGROUND_EVENTS

The example you have earlier is an extraordinary use that I don't think anyone would be using instead of if is_mouse_button_pressed(MouseButton::Left) { or if is_mouse_button_released(MouseButton::Left) { which both would be fixed by simply clearing input queue (to avoid sticky states). The only way clearing the input queue creates an issue is in your example. But I don't think it's a realistic example since I don't see the point in mirroring the input state of the engine instead of just reading the input state from the engine itself.

[profan]

@jonatino Right, well whatever the outcome is, as long as it results in a consistent observable behaviour for miniquad and macroquad applications where one half of an input event doesn't go missing (ie. any key pressed has an accompanying key released, or you end up with inconsistent game states or having to deal with the fact that you might never get a key released for a given key pressed).

I get your point about window focus and that generally unfocused windows don't usually get input events for obvious security reasons (and also it's probably just a mess if all applications would be capturing input events).

Overall I don't really care that much what we end up with, I just want the observable behaviour to not a little bit of a mess as it currently is (especially as you barely have window focus information in macroquad applications, so fixing the pressed/released state stuff is harder for someone who isn't working at the miniquad level).

Example of how Godot sorted this out: https://github.com/godotengine/godot/pull/28061

[InnocentusLime]

@jonatino Right, well whatever the outcome is, as long as it results in a consistent observable behaviour for miniquad and macroquad applications where one half of an input event doesn't go missing (ie. any key pressed has an accompanying key released, or you end up with inconsistent game states or having to deal with the fact that you might never get a key released for a given key pressed).

I get your point about window focus and that generally unfocused windows don't usually get input events for obvious security reasons (and also it's probably just a mess if all applications would be capturing input events).

Overall I don't really care that much what we end up with, I just want the observable behaviour to not a little bit of a mess as it currently is (especially as you barely have window focus information in macroquad applications, so fixing the pressed/released state stuff is harder for someone who isn't working at the miniquad level).

Example of how Godot sorted this out: godotengine/godot#28061

I decided to give that issue a shot :) It seems we would need both miniquad and macroquad patches. The miniquad patch can be found here: not-fl3/miniquad#552. I will test it on OsX a bit later, but if someone else tests it on their machines -- it would be greatly appreciated.

[jonatino]

@InnocentusLime I also started on this, just threw something together in the past 10mins because this bug was getting very annoying for my players.

Added a focus gained/lost event for the window.

Miniquad commits: windows & wasm: https://github.com/VoltaSoftware/miniquad/commit/59549b66aba7368a06731c4b901846983dabd5c6 android & linux: https://github.com/VoltaSoftware/miniquad/commit/5a87f41b5d0537c9a312511a9b209405c6481342 macos: https://github.com/VoltaSoftware/miniquad/commit/3e3263384a4975b51e6eabbcb79b89bdd7aea0e0 ios: https://github.com/VoltaSoftware/miniquad/commit/9a8af70ea3f3f37fbbd86ee81e84aca9f496cb45

Macroquad commit: https://github.com/VoltaSoftware/macroquad/commit/49972d9a16ce7b19cb08526ac16362138af70791?w=1#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759