librashader icon indicating copy to clipboard operation
librashader copied to clipboard

mtl_filter_chain_frame with (1, 1)-sized output texture can cause use of (0, 0)-sized intermediate MTLTextures, crash

Open jcm93 opened this issue 10 months ago • 0 comments

The old implementation of clear() in ares's Metal driver called into librashader with a regular-sized (e.g. 640x480— in any case, larger than 1x1) source texture and a 1x1 sized output texture. With some shaders, this caused an assertion failure and crash with librashader creating and trying to use (0, 0) sized MTLTextures.

Affected shaders include some in the koko-aio family, for example clean-scanlines-classic_take and probably others (full extent not known).

cc https://github.com/ares-emulator/ares/pull/1466

Possibly helpful trace:

Thread 20 Crashed::  Dispatch queue: com.ares.metal-renderer
0   libsystem_kernel.dylib                   0x1829f6a60 __pthread_kill + 8
1   libsystem_pthread.dylib                  0x182a2ec20 pthread_kill + 288
2   libsystem_c.dylib                        0x18293ba20 abort + 180
3   libsystem_c.dylib                        0x18293ad10 __assert_rtn + 284
4   Metal                                    0x18cee51a4 MTLReportFailure.cold.1 + 48
5   Metal                                    0x18cec1dc0 MTLReportFailure + 464
6   Metal                                    0x18ceb7d20 _MTLMessageContextEnd + 876
7   Metal                                    0x18cd85440 -[MTLTextureDescriptorInternal validateWithDevice:] + 2312
8   AGXMetalG13X                             0x1d79ee8ac -[AGXTexture initWithDevice:desc:isSuballocDisabled:] + 56
9   librashader.dylib                        0x13e6d71cc librashader_runtime_mtl::texture::OwnedTexture::new::hee3c3dffbb2dd244 + 468
10  librashader.dylib                        0x13e6df938 _$LT$librashader_runtime_mtl..texture..OwnedTexture$u20$as$u20$librashader_runtime..scaling..ScaleFramebuffer$GT$::scale::h0144f4733616fae9 + 416
11  librashader.dylib                        0x13e5bc958 libra_mtl_filter_chain_frame + 3780
12  ares                                     0x1011c1088 VideoMetal::outputHelper(unsigned int, unsigned int, id<MTLTexture>) + 564
13  libdispatch.dylib                        0x18287c750 _dispatch_call_block_and_release + 32
14  libdispatch.dylib                        0x18287e3e8 _dispatch_client_callout + 20
15  libdispatch.dylib                        0x182885a14 _dispatch_lane_serial_drain + 748
16  libdispatch.dylib                        0x182886544 _dispatch_lane_invoke + 380
17  libdispatch.dylib                        0x1828912d0 _dispatch_root_queue_drain_deferred_wlh + 288
18  libdispatch.dylib                        0x182890b44 _dispatch_workloop_worker_thread + 404
19  libsystem_pthread.dylib                  0x182a2b00c _pthread_wqthread + 288
20  libsystem_pthread.dylib                  0x182a29d28 start_wqthread + 8

jcm93 avatar May 01 '24 16:05 jcm93