update_with_buffer hangs on macOS while you resize the window

[LoganDark]

gif

code

use minifb::{Window, WindowOptions};

fn main() {
	let mut window = Window::new(
		"rust-raytracer",
		10,
		10,
		WindowOptions {
			resize: true,
			..WindowOptions::default()
		}
	).expect("Couldn't create window");

	while window.is_open() {
		let (width, height) = window.get_size();
		let mut buffer: Vec<u32> = vec![0xFFFFFF; width * height];

		buffer[width + 1] = 0xFF0000;

		window.update_with_buffer(&buffer, width, height).unwrap();
	}
}

Observe how the red pixel gets large while I drag to resize the window bigger, but as soon as I release the left click, the buffer updates and the red pixel returns to normal size

Is it possible to fix this?

[emoon]

That is indeed odd. I will investigate this evening.

[emoon]

I assume you want the pixel to stay small at all time or do you want the buffer itself to scale?

[LoganDark]

~~I am manually updating the buffer as fast as possible. Basically, while the window is resizing, it only displays the buffer that was there when the resize was started. But it should display the most up-to-date buffer even while the resize is still happening. That way, the pixel would stay the same size, rather than the buffer stretching like it is now.~~

~~At most the buffer should be 1 frame out of date (ideally zero).~~

[LoganDark]

~~My code still updates the buffer while the window is being resized but the window still stretches the old buffer instead of displaying the new one. That is the bug.~~ Ugh, I am really bad at explaining things

[emoon]

I get it

[LoganDark]

sorry, I just thought my own explanation was inadequate, not that you didn't understand it - hope it gets fixed soon :D

[emoon]

Ok. I remember what the problem is now. On macOS (for whatever reason) when the user starts to resize a window I don't get any callbacks out to my "main code" anymore (i.e where the user updates the the buffer size) so that is "frozen" until the user releases the button.

You can make it look slightly better by setting up your window this way

	let mut window = Window::new(
		"rust-raytracer",
		10,
		10,
		WindowOptions {
			resize: true,
			scale_mode: ScaleMode::UpperLeft,
			..WindowOptions::default()
		}
	).expect("Couldn't create window");

But until I can figure out why I don't get any updates anymore when the user start to resize the window I'm not sure how to solve this issue.

[LoganDark]

Well, that only really works if your application's purpose is to display a red pixel in the top left :P

For now, it doesn't look like there are any workarounds for the root issue (buffer not updating while the window is resizing) so I'll just have to wait until the issue is actually solved.

[LoganDark]

By the way, what do you mean callbacks to your main code? Do you mean the update_with_buffer call actually doesn't do anything? Or does it still update the buffer, but the native window just doesn't notice until after the resize?

[emoon]

if you add a println!(..) in your update code (just before update_with_buffer(..)) you will notice it won't get called while you are resizing the window. Inside the update_with_buffer I'm calling a macOS API call to updated events and one is "stuck" inside that function until the resize is complete.

[LoganDark]

you will notice it won't get called while you are resizing the window.

Oh wow, I was assuming that wasn't the case (which is why I initially said that my code was updating the buffer during resize). That's a bit weird...

[emoon]

Yeah I don't know how to solve it as macOS is not leaving the function. It might be possible to change the whole event update logic but I'm scared that it might require the current API to break (which I don't want to do)

[LoganDark]

Is it that new events simply aren't being generated during the resize? Or are they just backing up in the queue while the function hangs?

but I'm scared that it might require the current API to break (which I don't want to do)

I think it would be possible to just change the way you poll for events / deal with the event queue, since according to you some part of that is what's hanging. That would require no changes to the public API. Buuut, I have no idea what I'm talking about at all so take everything with a grain of salt.

[emoon]

Yeah it might be possible. It's just something I can't fix right now, sorry.

[LoganDark]

It's a real shame because this is the most complete framebuffer library I could find. I hope it does get fixed one day. Thank you for looking into it though!

[emoon]

Yeah the plan is to try to fix all resizing issues as there are some issues on other platforms as well. It's more of a PITA than I thought it would be :) TBH this lib has ended up being larger than my initial plan but I guess that is how things goes at times.

[john01dav]

It may be worth checking if this allows the issue to be fixed, by updating the buffer without polling input. Of course, this isn't a real solution, since resizing shouldn't break input polling either, but it could produce a workaround for some applications, and may help with debugging this.

[LoganDark]

@john01dav I suppose that could work if another thread polls for input, so the blocking wouldn't affect the main thread... Rust makes this pretty nice :P

[emoon]

It might be possible but I also know there are a bunch of restrictions of macOS APIs where they have to be done on the main thread (esp for UI) so some more research will be needed for this.

[LoganDark]

@emoon Wait, really? If that became a built-in feature, that would be great. Of course more research is needed, including of course whether or not the multi-threaded approach can even be done, and then after that how the API would have to be changed, then what sort of concurrent approach to take (i.e. maybe the main thread should take snapshots of all the current inputs? but, maybe it would miss inputs that way, so maybe you'd have to take an event-based approach, etc etc...) and so on...

[emoon]

Yeah this is somewhat of a tricky topic. It's esp makes it a bit more tricky as minifb currently fits my needs (and goes beyond) so while it's great that people like to use it, I need figure out how much work I'm willing to put into fixing this. It's may sound a bit harsh, but I need to make a trade-off between other projects I'm working on as well.

[LoganDark]

@emoon It's still helpful to discuss so that someone (like me) can open a pull request one day.

[emoon]

Agreed. Problem with minifb also is that as it's cross-platform the same API needs to work on all platforms and needs to be changed for all of them. It makes the work even more complex, but I'm sure it's doable to figure something out.

[LoganDark]

I can cross-compile for all three major platforms (macOS, Linux and Windows). It wouldn't be too hard to test compilation on each platform, but testing if it works is another matter (namely, I don't have a Linux system with a desktop environment, but I might soon once I add one to my Arch installation).

[emoon]

Yeah. The thing is that one usually needs to implement for all the platforms and verify it working as well.

[parasyte]

FWIW, the issue with this one is explained in detail in https://github.com/rust-windowing/winit/issues/219#issuecomment-315830359 and https://github.com/glfw/glfw/issues/1251

[LoganDark]

@emoon You might want to check this out: https://github.com/kovidgoyal/kitty/commit/3f9d04b616bcce85ebfa319db482d888ca09b1fb

[SidneyNemzer]

I think I'm seeing this same behavior on windows.

I was expecting the loop to continue updating the buffer while resizing.

Code (sorry there's a bit of extra code, let me know if it would help to make this more minimal)

[LoganDark]

One of these years I'll get to finishing and releasing glfb2 which will integrate with winit and not have this problem. But of course wgpu has one-frame lag on macOS and (presumably) other platforms as well, which is just great.

[emoon]

I think in order to solve this quite a bit of the internals of minifb has to be changed on how it drives the update loop. I'm not sure if it's worth it or not.

I have considered using winit in the past, but the problem is that winit it isn't really that "mini" and has quite a bunch of dependencies and I want minifb to be fast to compile and easy to use (no need for setting up extra callbacks for event handling)