Display issues on MacOS

[whmountains]

As discussed in #4 I'm working on getting Azul to run on MacOS. There were no problems with the build but I currently get a blank screen when I try to load an SVG.

At first every time I loaded an SVG the program would panic with version 130 is not supported. I tried switching to #version 140 since that's what the glium examples use and sure enough, no more crashes.

What I have now is:

• App window flashes white at startup, then turns black.
• Clicking the black window opens a file picker.
• I choose tiger.svg from the repository.
• App window turns white, and the terminal logs 5 identical messages about pan and zoom.

I'm not sure if the shaders need to be updated to support version 140, or if the bug lies elsewhere. I'm not super familiar with graphics programming so any thoughts would be appreciated.

[fschutt]

Thanks for trying it out.

It is strange because technically if version 140 is supported, then 130 is supported, too, but macOS is a bit special, see: https://stackoverflow.com/questions/20264814/glsl-version-130-on-mac-os-x-causes-error - on macOS it seems that you have to select a core profile, otherwise, macOS doesn't load anything newer than OpenGL 2.1.

Could you run glxinfo (I think it's built-in on Macs) to output what your graphics driver / supported OpenGL version is?

You shouldn't see a black window on startup, it should be a grey-ish rectangle with a slight gradient. Could you post a screenshot of what you get when you just run cargo run --example debug? Another thing - could you try running any of the webrender examples from here - do they produce a black screen, too?

Do you actually see the tiger on the screen (at least for one frame)? If you resize the window or scroll, the tiger will probably disappear, yes. This is related to this OpenGL bug I found in webrender / glium - I'll fix that today. If you don't see the tiger then it could be because of the core profile problem.

The logging messages about pan and zoom are OK, they are removed now.

[fschutt]

@whmountains Could you try again (I just pushed an update), I think that at least the crash on SVG loading should be fixed now. I set the minimum version to #150, since that is the minimum that webrender itself supports. Other than that, I have made sure that the OpenGL core profile is selected. So you should at least see the tiger for one frame (don't resize the window after you've loaded the SVG) and the example shouldn't crash when running.

[whmountains]

Example doesn't crash anymore. Yeah!

Here is a short video showing the launch sequence. I don't see the tiger for even one frame, but maybe it's still caused by the same underlying bug.

[fschutt]

Could you run any of the webrender examples? What's weird to me is that it shows a black screen on startup - most webrender developers use macs themselves and the initial draw doesn't use any fancy shaders, only stock webrender stuff - so it should behave normally and show a white / grey button that spans the whole window.

Because if you have problems running the webrender examples, then it's a bug in webrender or your current graphics driver and should be reported there, if not then it's a bug in azul.

[fschutt]

For reference, this is what you should see on startup:

[whmountains]

I just tested most of the webrender examples and they run fine. 🤔

[fschutt]

A theory I have is that it is an updating bug. Meaning the contents do get drawn, but they never get swapped to the window. Try this: move this line: https://github.com/maps4print/azul/blob/master/src/app.rs#L187 out of the if statement and at the top of the window loop (so that it updates the window every 16ms instead of only when necessary). Then try running it again, if you at least see the grey background.

If you see the grey square, this means that the update event doesn't get submitted correctly on macOS.

If that still doesn't work, try disabling the if statement (i.e. set it to if true) in this line: https://github.com/maps4print/azul/blob/master/src/app.rs#L216 - so that it re-renders the screen always (every 16ms), no matter what. Because if you can run the webrender examples, that means that the rendering is done correctly, just the updating part may hang.

[whmountains]

Sounds good. I hope to have time to try your suggestions tomorrow.

[fschutt]

@whmountains Did it work?

[whmountains]

After some fiddling I realized that I can get the gray square to display by resizing the window. The layout is all messed up but it works! This is true whether or not I move the swap buffers command to the top of the loop or short-circuit the update logic.

EDIT: If I move the window between monitors it will correctly draw the gray box with rounded corners with no layout issues and no flickering. Go figure!

[whmountains]

To summarize, forcing an update every 16ms doesn't seem to help. What seems to work is the following:

• Moving the window between screens. Works every time, no layout issues.
• Resizing the window. Causes flickering and artifacts. Usually the rectangle doens't fill the new window size.
• Running the example without an external monitor.

More about running without an external monitor: My MacBook Pro has both Intel HD 4000 integrated graphics and Nvidia GeForce 750M dedicated graphics and automatically switches between them to save power. An external display will always use dedicated graphics, while integrated graphics is more likely while using the HD 4000. Running the azul causes MacOS to decide I must need the dedicated GPU and so it hot-switches. My theory is that change-over somehow example triggers the re-draw we need.

[fschutt]

Hmm... what happens if you set the rendering to software rendering - in the debug example, see the WindowCreateOptions::default() - set it to: WindowCreateOptions { renderer_type: RendererType::Software, .. Default::default() } - this should force it to use the built-in OSMesa software renderer. Do you still see the bugs / black display then? If yes, it's a relayout / rerendering bug, otherwise it's probably a GPU driver bug.

Does it still show the black screen on startup? We can do the layout later, that's probably a HiDPI issue or a mathematical bug. I just want to avoid having a black screen on startup - if you resize the window, azul calls the layout function. So if you have a black screen that means that the layout function doesn't get called correctly. Because azul wants to be efficient, it doesn't call it every frame, but this can lead to bugs like this where it's "too conservative" and doesn't render at all because it thinks the screen hasn't changed.

If you move the window between screens, there is an event being emitted, I think it's HiDPIFactorChanged - could you insert a println!("{:?}", frame_event_info); here: https://github.com/maps4print/azul/blob/master/src/app.rs#L215

What happens if you leave the swap buffers command where it is, but turn the other statement (on line 216, https://github.com/maps4print/azul/blob/master/src/app.rs#L216) to if true? Does it still display a black screen on startup?

Also, regarding the tiger, I fixed that bug yesterday - could you update (git pull) and try again, if the tiger loads and gets rendered (it should display now and scrolling the tiger should work)? Thanks for going through this, I didn't expect this to be that different, but cross-platform is always difficult.

[fschutt]

When resizing, is the rectangle just lagging behind (i.e. "catching up" with the window size during resizing)? If yes, then that's to be expected when running in debug mode - try running it in --release mode.

The flickering could be because of the 16ms timer - the hardware clock (when the GPU actually swaps the buffer) and the 16ms can become misaligned. There are ways to avoid this, but I'm not entirely sure on how to synchronize these two.

[whmountains]

Switching to software rendering doesn't change anything. It still shows a black screen until I move the window to another monitor.

To help me separate signal from noise in the window events I wrote a little helper function that omits the cursor position field and only logs the struct fields that are different from the default.

When I resize the window I get these events: (omitting fields un-changed from their default value):

FrameEventInfo {
  should_redraw_window: true,
  should_hittest: true,
  new_window_size: Some((765, 553)),
  is_resize_event: true
}
FrameEventInfo { should_swap_window: true, should_hittest: true }
FrameEventInfo { should_swap_window: true, should_hittest: true }

I tested it twice and got exactly three events both times.

When I move to a new monitor, I get this event:

FrameEventInfo {
  should_redraw_window: true,
  new_window_size: Some((800, 600)),
  is_resize_event: true
}

I guess I don't get the HiDPIFactorChanged event because neither my MacBook Pro display nor my external monitor are HiDPI. I have the last model of MacBook Pro before Apple introduced "retina" displays.

When switching from integrated to dedicated GPU I get this sequence of events:

FrameEventInfo {
  should_redraw_window: true,
  new_window_size: Some((800, 600)),
  is_resize_event: true
}
FrameEventInfo { should_swap_window: true }
FrameEventInfo { should_swap_window: true }

I will continue testing and keep you updated.

[whmountains]

Adding || true to if statement on line 216 of app.rs causes it to draw correctly.

[whmountains]

EDIT: removed comment because it was incorrect.

[andresobral]

Hi, I am following this project too and I want to help on what is possible. I have a macbook pro retina display and I tested the example and the following happened:

• It starts with a black screen;
• If you resize it, it flicks between grey and black and when you stop resizing, the button appears;
• If you resize more than the original window size, a black stripe appears;
• The tiger svg does not work at all;

I hope it helps on something.

[whmountains]

I can get the initial re-draw to work by changing https://github.com/maps4print/azul/blob/2a9e4ce4866d3473df7ffc72cd58ec1056bb45c6/src/app.rs#L146 from this:

let mut force_redraw_cache = vec![0_usize; self.windows.len()];

to this:

let mut force_redraw_cache = vec![1_usize; self.windows.len()];

That is, force an initial redraw within the event loop for each window. 'window_loop is doing something that makes the redraw succeed where do_first_redraw failed.

[whmountains]

Just opened #7 which seems to cleanly fix the problem. Unfortunately I still don't know the root cause, but it most be somehow related to difference between the code that runs in the 'window_loop and the code that runs in the do_first_redraw function.

If you'd like me to run some more tests to get to the bottom of this just let me know.

[whmountains]

Sorry, didn't mean to click the "close" button.

[fschutt]

Right, so at least it's not a GPU bug, it's a bug with the redraw function. That would be way harder to fix and debug. The events look normal to me. And since you said that resizing itself works (flickering, but that's another topic) and calling update more often doesn't help, I assume that it has more to do with the rendering function.

It's rather hard debugging this remotely without a Mac, I'd need to buy macOS somewhere and use it in a VM. If you still want to debug this, try logging the window size and look at preprocess_event, what events actually get emitted.

@whmountains ~~I've pushed an update in 4662e13abfe76ddb225fa76cf19a3e439495f286 - could you try resetting it back to 0 and trying with the new update? I basically noticed that do_first_redraw doesn't notify webrender of the initial window size, that's the only thing I changed now.~~

EDIT: #7 is a better idea anyways, since it reduces the duplicated code.

Does it work now if you set force_redraw_cache to 1 - does it still show black during resizing or does the resizing work now?

Also thanks @andresobral for testing.

[whmountains]

do_first_redraw doesn't notify webrender of the initial window size

That was my first guess as well, but no such luck. I tried again with your latest commit just to make sure.

Does it work now if you set force_redraw_cache to 1 - does it still show black during resizing or does the resizing work now?

No. There still seems to be layout issues with resizing, just as @andresobral described. Here's a video to give you an idea: https://imgur.com/a/JQRgiod.

[fschutt]

Well okay... I think #7 is a good idea (I'll merge it after the tests pass), but what do you see now when you run the app? Does the resizing work? Do you still see any black? Does the tiger work for you?

[whmountains]

Sorry, I edited my comment just as you were posting yours which may have been confusing. I still see black while re-sizing. I just get a white screen after loading the tiger.

Thank you for your patience helping those of us with macs debug this. I will look a bit longer to see if I can find the root cause, but if nothing turns up I'll move on to the layout and svg issues.

[andresobral]

Tested here and unfortunately nothing has changed. The black screen continues and the tiger does not appear.

I've changed force_redraw_cache to 1 and the only thing that changed was the position of the button.

[whmountains]

@andresobral Just to clarify, you still see a black screen at startup until you resize. Is that correct?

Can you confirm that you're initializing all values of force_redraw_cache to 1 like I did in https://github.com/whmountains/azul/blob/6464915503412bd29986727581d52f02ced27263/src/app.rs#L145 and not modifying it somewhere else? Sorry if this is obvious, I just want to make sure we're talking about the same thing.

Finally, do you mind sharing the specific model of Mac you have, or at least which GPU? I have a MacBook Pro non-retina 15-inch Mid 2012 with NVIDIA GeForce GT 650M 1024 MB dedicated GPU and an Intel HD Graphics 4000 1536 MB integrated GPU.

[fschutt]

Sorry, not actually closed, this was just because #7 contained the word "fixed"

[andresobral]

@whmountains You are correct, sorry, I had changed somewhere else before. If I changed the line that you said, I don't see the black screen on start up anymore.

My mac specifications: MacBook Pro retina 13-inch Early 2015 with Intel Iris Graphics 6100 1536 MB

[whmountains]

this was just because #7 contained the word "fixed"

Sorry, my bad.

MacBook Pro retina 13-inch Early 2015 with Intel Iris Graphics 6100 1536 MB

Thank you for that! So the issues at hand are probably more related to the MacOS platform and less with the fact that I have a really old GPU.

[fschutt]

@whmountains Thanks for the GIF, that makes it much easier to see what's going on. At least the initial draw is not a black screen, which is at least a bit of progress.

Yes, it's definitely not a GPU bug, just a bug with handling the resize events from MacOS correctly. What you're seeing while resizing the window is just GPU garbage, not actually azul drawing anything.

What I think is going on is that webrender does not get notified that the screen size has changed. In the app.rs render() function (at the bottom, line 589), try logging the various sizes, whether the window size has changed, etc. If anything look sketchy (i.e. a value that stays at (800, 600)) or has_window_size_changed has the wrong value, that would be a good indicator of a bug.

Essentially, azul generates (via the into_display_list_builder) a display list, which contains all the rectangles and things you see on the screen. Try to log the size of that display list via builder.content_size() (line 610). And log other things, like the incoming events (see the preprocess_event function), etc.

It takes a bit of experimentation, but some value should be wrong. I can't really help with this much in my position, sadly.

I'll try to fix the SVG shader, while it works on Windows and Linux, macOS seems to need a bit special care (possibly around floating point precision, etc.). I'll try to read the webrender shaders in the meantime if I can get anything from them regarding special Mac OpenGL problems.

[andresobral]

It makes sense that the webrender does not get notified on resize, because when you resize the window above the original size, a black stripe appears as shown in my previous comment. I will try to investigate too, based on what you said, if I can figure out something.

[fschutt]

Aside from that, I updated the shaders now - could someone try again if the tiger appears now? Thanks.

[andresobral]

Just tested here and unfortunately, the tiger does not appear. The behavior stays the same, the windows starts white, I have to resize a bit for the button to appear and when I select the tiger, nothing happens.

[zaynetro]

Thanks for the excellent work done so far!

I've tried running basic example from the readme on macos but the texts are not visible on the screen. My machine: MacBook Pro (Retina, 13-inch, Mid 2014) Intel Iris 1536 MB

I have no knowledge about font rendering. Although, I am willing to try out some tips on how to solve this and report the results.

---

Running the debug example crashes the application after pressing the button.

$ cargo run --release --example debug --features="svg_parsing"
    Finished release [optimized] target(s) in 21.23s
     Running `target/release/examples/debug`
[ERROR][azul::logging] We are sorry, an unexpected panic ocurred, the program has to exit.
Please report this error and attach the log file found in the directory of the executable.

The error ocurred in: libcore/option.rs at line 345 in thread main

Error information:
called `Option::unwrap()` on a `None` value

Backtrace:

backtrace::backtrace::trace
<backtrace::capture::Backtrace as core::default::Default>::default
backtrace::capture::Backtrace::new
azul::logging::set_up_panic_hooks::panic_fn
std::panicking::rust_panic_with_hook
std::panicking::continue_panic_fmt
core::panicking::panic_fmt
core::panicking::panic
debug::my_button_click_handler
<azul::app::App<T>>::run_inner
<azul::app::App<T>>::run
debug::main
std::rt::lang_start::{{closure}}
std::panicking::try::do_call
std::rt::lang_start_internal

I figured that this was due to sans-serif font not present on my machine. After changing the font to arial it worked until SVG rendering.

UPD: I didn't need to resize the window for the button to appear as mentioned in the previous comment. The button was already there.

SVG file used: https://en.wikipedia.org/wiki/Scalable_Vector_Graphics#/media/File:SVG_logo.svg

$ RUST_BACKTRACE=1 cargo run --release --example debug --features="svg_parsing"
    Finished release [optimized] target(s) in 20.16s
     Running `target/release/examples/debug`
objc[65182]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fffa9494c90) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x1289a9cd8). One of the two will be used. Which one is undefined.

The output seems to be related to file dialogue than to azul.

Let me know if you need more details!

[OtaK]

Is the whole thing running fine in HiDPI contexts? "macOS" bugs were mostly related to HiDPI since macs nowadays are all HiDPI screens and having a second screen makes weird things like dynamically switching DPI resolutions at runtime when switching screens. Other OpenGL GUI libraries (like Conrod) had lots of trouble figuring these things out.