filament icon indicating copy to clipboard operation
filament copied to clipboard

Scissor rect validation failed assertion on Metal

Open kpeeters opened this issue 2 years ago • 2 comments

I am getting 'Scissor rect validation failed assertion' errors with what seem to be uninitialised integer values for some of the rect components on Metal, since https://github.com/google/filament/pull/5644 landed. E.g.

-[MTLDebugRenderCommandEncoder setScissorRect:]:3635: failed assertion `Set Scissor Rect Validation
(rect.x(18446744073709551615) + rect.width(0))(18446744073709551615) must be <= render pass width(2732)

(added: that's of course a '-1', not a random value, but still looks like something did not get initialised beyond that). This is without actually ever calling setScissor anywhere. Am I supposed to call this with the same values as the Viewport when I don't want to make use of scissoring?

kpeeters avatar Aug 06 '22 11:08 kpeeters

This seems to be related to https://github.com/google/filament/issues/5892: the scissor rect validation assertion passes as long as the viewport does not extend beyond the screen/framebuffer size.

kpeeters avatar Aug 06 '22 18:08 kpeeters

Lacking time to do more advanced debugging, I stuck in some logging in MetalDriver.mm in the bit that starts with the comment 'Set scissor-rectangle.'. The first few frames it shows

sleft = 0.000000, sright = 2147483648.000000, stop = 0.000000, sbottom = 2147483648.000000
vleft = 0.000000, vright = 1512.000000, vtop = 0.000000, vbottom = 1008.000000
left = 0.000000, right = 1512.000000, top = 0.000000, bottom = 1008.000000

which looks ok (assuming the large values for sright and sbottom are intentional). This does not trigger the problem yet. Then at some point the assertion triggers, and the values for that frame are

sleft = 0.000000, sright = 2147483648.000000, stop = 0.000000, sbottom = 2147483648.000000
vleft = 18446744073709551616.000000, vright = 18446744073709551616.000000, vtop = 0.000000, vbottom = 2008.000000
left = 18446744073709551616.000000, right = 2732.000000, top = 0.000000, bottom = 2008.000000

I don't think I am changing the viewport to have vleft=vright=LONG_LONG_MAX, but I'll triple check.

(added: this is with https://github.com/google/filament/pull/5896).

kpeeters avatar Aug 08 '22 19:08 kpeeters

@kpeeters Thanks for your patience, I was out last week. I'll give this a closer look.

bejado avatar Aug 15 '22 17:08 bejado

@bejado No problem at all. I can confirm that I only call setViewport once, with sensible values.

kpeeters avatar Aug 15 '22 21:08 kpeeters

@kpeeters Thank you for that detailed debugging info. After further investigation, I suspect that your app (or Filament) is passing in a negative Viewport origin, which is valid, but not currently handled correctly.

I've updated https://github.com/google/filament/pull/5896 to fix this, which should now fix this issue and https://github.com/google/filament/issues/5892.

bejado avatar Aug 17 '22 04:08 bejado

@bejado All good now (for both issues). Yes, I was indeed using a negative Viewport origin. Thanks a lot, I can run HEAD again.

kpeeters avatar Aug 17 '22 08:08 kpeeters

@kpeeters Good to hear, apologies for the trouble!

bejado avatar Aug 17 '22 17:08 bejado

I am still seeing an assert failing when the left boundary of the scissor is beyond half of the viewport width. The assert shows a number for the viewport width which is half its actual size. Does not happen with the top being beyond half the viewport height, strangely. May have something to do with the fact that this device has UIScreen.main.nativeScale == 2, but I don't have any others available to test that. If there's something obvious that comes to mind, let me know, otherwise I'll do some more digging.

kpeeters avatar Sep 01 '22 07:09 kpeeters

I don't see any assert when adjusting the left boundary of the scissor beyond half the viewport width.

We have some test cases for scissor / viewport which you can play around with and run via:

./build.sh -p desktop debug backend_test_mac
out/cmake-debug/filament/backend/backend_test_mac --api metal '--gtest_filter=BackendTest.ScissorViewport*'

If you're able to reproduce the assertion please feel free to file another issue.

bejado avatar Sep 02 '22 18:09 bejado