update_with_buffer hangs on macOS while you resize the window

[LoganDark]

gif

code

use minifb::{Window, WindowOptions};

fn main() {
	let mut window = Window::new(
		"rust-raytracer",
		10,
		10,
		WindowOptions {
			resize: true,
			..WindowOptions::default()
		}
	).expect("Couldn't create window");

	while window.is_open() {
		let (width, height) = window.get_size();
		let mut buffer: Vec<u32> = vec![0xFFFFFF; width * height];

		buffer[width + 1] = 0xFF0000;

		window.update_with_buffer(&buffer, width, height).unwrap();
	}
}

Observe how the red pixel gets large while I drag to resize the window bigger, but as soon as I release the left click, the buffer updates and the red pixel returns to normal size

Is it possible to fix this?

[emoon]

That is indeed odd. I will investigate this evening.

[emoon]

I assume you want the pixel to stay small at all time or do you want the buffer itself to scale?

[LoganDark]

I am manually updating the buffer as fast as possible. Basically, while the window is resizing, it only displays the buffer that was there when the resize was started. But it should display the most up-to-date buffer even while the resize is still happening. That way, the pixel would stay the same size, rather than the buffer stretching like it is now.

At most the buffer should be 1 frame out of date (ideally zero).

[LoganDark]

My code still updates the buffer while the window is being resized but the window still stretches the old buffer instead of displaying the new one. That is the bug. Ugh, I am really bad at explaining things

[emoon]

I get it

[LoganDark]

sorry, I just thought my own explanation was inadequate, not that you didn't understand it - hope it gets fixed soon :D

[emoon]

Ok. I remember what the problem is now. On macOS (for whatever reason) when the user starts to resize a window I don't get any callbacks out to my "main code" anymore (i.e where the user updates the the buffer size) so that is "frozen" until the user releases the button.

You can make it look slightly better by setting up your window this way

	let mut window = Window::new(
		"rust-raytracer",
		10,
		10,
		WindowOptions {
			resize: true,
			scale_mode: ScaleMode::UpperLeft,
			..WindowOptions::default()
		}
	).expect("Couldn't create window");

But until I can figure out why I don't get any updates anymore when the user start to resize the window I'm not sure how to solve this issue.

[LoganDark]

Well, that only really works if your application's purpose is to display a red pixel in the top left :P

For now, it doesn't look like there are any workarounds for the root issue (buffer not updating while the window is resizing) so I'll just have to wait until the issue is actually solved.

[LoganDark]

By the way, what do you mean callbacks to your main code? Do you mean the update_with_buffer call actually doesn't do anything? Or does it still update the buffer, but the native window just doesn't notice until after the resize?

[emoon]

if you add a println!(..) in your update code (just before update_with_buffer(..)) you will notice it won't get called while you are resizing the window. Inside the update_with_buffer I'm calling a macOS API call to updated events and one is "stuck" inside that function until the resize is complete.

[LoganDark]

you will notice it won't get called while you are resizing the window.

Oh wow, I was assuming that wasn't the case (which is why I initially said that my code was updating the buffer during resize). That's a bit weird...

[emoon]

Yeah I don't know how to solve it as macOS is not leaving the function. It might be possible to change the whole event update logic but I'm scared that it might require the current API to break (which I don't want to do)

[LoganDark]

Is it that new events simply aren't being generated during the resize? Or are they just backing up in the queue while the function hangs?

but I'm scared that it might require the current API to break (which I don't want to do)

I think it would be possible to just change the way you poll for events / deal with the event queue, since according to you some part of that is what's hanging. That would require no changes to the public API. Buuut, I have no idea what I'm talking about at all so take everything with a grain of salt.

[emoon]

Yeah it might be possible. It's just something I can't fix right now, sorry.

[LoganDark]

It's a real shame because this is the most complete framebuffer library I could find. I hope it does get fixed one day. Thank you for looking into it though!

[emoon]

Yeah the plan is to try to fix all resizing issues as there are some issues on other platforms as well. It's more of a PITA than I thought it would be :) TBH this lib has ended up being larger than my initial plan but I guess that is how things goes at times.

[john01dav]

It may be worth checking if this allows the issue to be fixed, by updating the buffer without polling input. Of course, this isn't a real solution, since resizing shouldn't break input polling either, but it could produce a workaround for some applications, and may help with debugging this.

[LoganDark]

@john01dav I suppose that could work if another thread polls for input, so the blocking wouldn't affect the main thread... Rust makes this pretty nice :P

[emoon]

It might be possible but I also know there are a bunch of restrictions of macOS APIs where they have to be done on the main thread (esp for UI) so some more research will be needed for this.

[LoganDark]

@emoon Wait, really? If that became a built-in feature, that would be great. Of course more research is needed, including of course whether or not the multi-threaded approach can even be done, and then after that how the API would have to be changed, then what sort of concurrent approach to take (i.e. maybe the main thread should take snapshots of all the current inputs? but, maybe it would miss inputs that way, so maybe you'd have to take an event-based approach, etc etc...) and so on...

[emoon]

Yeah this is somewhat of a tricky topic. It's esp makes it a bit more tricky as minifb currently fits my needs (and goes beyond) so while it's great that people like to use it, I need figure out how much work I'm willing to put into fixing this. It's may sound a bit harsh, but I need to make a trade-off between other projects I'm working on as well.

[LoganDark]

@emoon It's still helpful to discuss so that someone (like me) can open a pull request one day.

[emoon]

Agreed. Problem with minifb also is that as it's cross-platform the same API needs to work on all platforms and needs to be changed for all of them. It makes the work even more complex, but I'm sure it's doable to figure something out.

[LoganDark]

I can cross-compile for all three major platforms (macOS, Linux and Windows). It wouldn't be too hard to test compilation on each platform, but testing if it works is another matter (namely, I don't have a Linux system with a desktop environment, but I might soon once I add one to my Arch installation).

[emoon]

Yeah. The thing is that one usually needs to implement for all the platforms and verify it working as well.

[parasyte]

FWIW, the issue with this one is explained in detail in rust-windowing/winit#219 (comment) and glfw/glfw#1251

[LoganDark]

@emoon You might want to check this out: kovidgoyal/kitty@3f9d04b

[SidneyNemzer]

I think I'm seeing this same behavior on windows.

I was expecting the loop to continue updating the buffer while resizing.

Code (sorry there's a bit of extra code, let me know if it would help to make this more minimal)

[LoganDark]

One of these years I'll get to finishing and releasing glfb2 which will integrate with winit and not have this problem. But of course wgpu has one-frame lag on macOS and (presumably) other platforms as well, which is just great.

[emoon]

I think in order to solve this quite a bit of the internals of minifb has to be changed on how it drives the update loop. I'm not sure if it's worth it or not.

I have considered using winit in the past, but the problem is that winit it isn't really that "mini" and has quite a bunch of dependencies and I want minifb to be fast to compile and easy to use (no need for setting up extra callbacks for event handling)

[LoganDark]

I think in order to solve this quite a bit of the internals of minifb has to be changed on how it drives the update loop. I'm not sure if it's worth it or not.

AFAICT, this issue cannot feasibly be solved without moving away from polling which is like 80% of minifb's API. Context switching / coroutines / whatever are not reliable or portable enough to use here either unfortunately.

I have considered using winit in the past, but the problem is that winit it isn't really that "mini" and has quite a bunch of dependencies and I want minifb to be fast to compile and easy to use (no need for setting up extra callbacks for event handling)

minifb is good at what it does. It gets a buffer on screen and lets you poll for some basic inputs. But yeah, this would require a huge shift in minifb's API and probably isn't worth it, since that isn't what this crate is for.

[emoon]

Agreed. So I think the only way would be to try to run the event loop on a separate thread and when the application checks for input etc it will get buffered input from that thread. Now having a thread has number of issues as I think that macOS for example only allows some APIs (regarding to windows and such) can only happen on the main-thread.

And of course adding threading (even hidden from the user) adds quite a bit of extra code for each backend and will make the code overall more complicated to deal with.. so hum 🤔

[LoganDark]

Now having a thread has number of issues as I think that macOS for example only allows some APIs (regarding to windows and such) can only happen on the main-thread.

It's possible to flip this around and have the main thread be controlled by minifb. Multi-window is already not really a supported use case.

[emoon]

Some people do use multi-windows tho so that would break it for them... :/ So I'm not really sure what the best way would be without breaking the whole API. Maybe it would be possible to just have a separate polling for window size or something like that which is the biggest problem here 🤔

[emoon]

Tho that doesn't help the macOS case that doesn't get any update during resize

[LoganDark]

Some people do use multi-windows tho so that would break it for them... :/

I didn't even know that was possible. Wouldn't multi window have multiple event loops or something that would break completely on OSes like macOS?

[emoon]

I just tested the example https://github.com/emoon/rust_minifb/blob/master/examples/multi.rs under macOS and it runs fine at least

[emoon]

That said it doesn't do very much :)

[LoganDark]

I just tested the example https://github.com/emoon/rust_minifb/blob/master/examples/multi.rs under macOS and it runs fine at least

Interesting, didn't even know that was an example. Anyway, if you ever do want to try implementing the "thread notification" approach, you would probably have to separate event loops and windows.

You'd create your event loop, create your windows on that event loop, specify a... callback function... for each window (D::::) and then run the event loop which will block the main thread forever.

At least, that's how I'd imagine it working.

It's worth looking into, but not worth implementing right now imho. After all, minifb is not broke :)

[emoon]

Thanks! and agreed. I will perhaps experimint with it some day when I feel up to it and see if I can come up with something that would work.

[LoganDark]

It hasn't been mentioned here yet so here goes: mini_gl_fb is an alternative to minifb that uses winit for event handling. It does not freeze on resize, but it depends on OpenGL and glutin and stuff so it's definitely a lot heavier. I'm also not sure if it's good for heavily interactive projects because of the event-based approach - there is a game of life example though. feature matrix

It's an unpopular library with a quite small userbase, but it's definitely an option somewhere between minifb and wgpu.

[emoon]

Thanks! That is good to know. My initial vision for minifb was just an simple library to be used for experemints when wanting to plot some pixels (the initial api only had open window with fixed size check for esc and close window) over the years it grew into a more feature rich api that supporting resizing, menus, etc.

Oh well, I'm happy people still find it useful even if it has it's sets of issues :)

[superosku]

Hi! I am having the same issue on macOS but it happens any time a move the window or drag&drop on the window. Is there any workaraund for that?

I am trying to do a simple game with rust, minifb and raqote so drag and dropping is a must feature.

[emoon]

Hi,

No there is currently no workaround for it.