GfxExp icon indicating copy to clipboard operation
GfxExp copied to clipboard

Crash in path_tracing project in debug mode only

Open Hurleyworks opened this issue 3 years ago • 7 comments

Hi,

I'm getting a consistent crash in all scenes I try using the path_tracing project in Debug mode only. Release mode works fine. The error message that appears using Nsight->CUDA Debugging is [Error 700: Illegal address during kernel execution].

And the debugger usually stops at this line

CUDA_DEVICE_KERNEL void RT_CH_NAME(pathTraceBaseline)() {
    pathTrace_closestHit_generic();
}

OptiX 7.40 / CUDA 11.7 Driver: 516.59 Visual Studio 2022 17.2.4 Windows 10 RTX 2070 SUPER

Can you confirm this using your environment? I wonder if it has anything to do with this thread. https://forums.developer.nvidia.com/t/illegal-address-error-when-using-both-geometrytriangles-and-geometry-nodes/218956/18

Hurleyworks avatar Jul 21 '22 23:07 Hurleyworks

Thanks for the report. I'm a bit busy now, I hope I can check the issue soon.

shocker-0x15 avatar Jul 24 '22 09:07 shocker-0x15

Okay, no worries. :)

On Sun, Jul 24, 2022 at 5:16 AM shocker-0x15 @.***> wrote:

Thanks for the report. I'm a bit busy now, I hope I can check the issue soon.

— Reply to this email directly, view it on GitHub https://github.com/shocker-0x15/GfxExp/issues/6#issuecomment-1193278734, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB56PPYFT7MOHMNLSRRAYMDVVUCWRANCNFSM54JJKJYQ . You are receiving this because you authored the thread.Message ID: @.***>

-- -Steve http://berserko.hurleyworks.com/ http://www.hurleyworks.com

Hurleyworks avatar Jul 25 '22 22:07 Hurleyworks

I have probably met a similar issue on another program, so I tried to see optixu as well, and I confirmed the crash only in Debug mode.

  • I saw the crash happens even with the same ptx as Release.
  • When I changed several OptiX options that are different between Debug and Release to use the Release's ones, the issue disappeared.

This looks something fishy on OptiX-side. If we could create a decently small repro, it is good to report to NV with it. But for now using Release-like options for OptiX can be a workaround. Recently, OptiX 7.5 introduced a new feature for better debugging (OptiX-IR instead of PTX), but that feature itself is still not so reliable and the latest driver seems suspicious as in this issue as well... This is an ironic situation. However we are still able to use printf debug :)

Thanks for letting me know the forum thread. Let me keep this issue opened until we find a proper way to fix.

shocker-0x15 avatar Aug 04 '22 17:08 shocker-0x15

I still haven't been able to create simpler reproducer but I did make a nice render using the path_tracing code. :) empty_shells

Hurleyworks avatar Aug 24 '22 10:08 Hurleyworks

Wow, high-fidelity! Did you use PBR textures (base color, roughness, metallic, normal)?

shocker-0x15 avatar Aug 24 '22 16:08 shocker-0x15

Yes, I used the SimplePBR material and just environment lighting from https://polyhaven.com/a/cape_hill

Thanks again for making your work available to use and learn from.! :)

On Wed, Aug 24, 2022 at 12:32 PM shocker-0x15 @.***> wrote:

Wow, high-fidelity! Did you use PBR textures (base color, roughness, metallic, normal)?

— Reply to this email directly, view it on GitHub https://github.com/shocker-0x15/GfxExp/issues/6#issuecomment-1225962346, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB56PP3EGES7CKUF4PX3YS3V2ZFAPANCNFSM54JJKJYQ . You are receiving this because you authored the thread.Message ID: @.***>

-- -Steve http://berserko.hurleyworks.com/ http://www.hurleyworks.com

Hurleyworks avatar Aug 24 '22 17:08 Hurleyworks

Okay, thanks for confirming and giving a workaround. I'll see if I can make a simpler example. Unfortunately I couldn't find any similar crashes in OptiX_Utility.

On Thu, Aug 4, 2022 at 1:33 PM shocker-0x15 @.***> wrote:

I have probably met a similar issue on another program, so I tried to see optixu as well, and I confirmed the crash only in Debug mode.

  • I saw the crash happens even with the same ptx as Release.
  • When I changed several OptiX options that different between Debug and Release to use the Release's ones, the issue disappeared.

This looks something fishy on OptiX-side. If we could create a decently small repro, it is good to report to NV with it. But for now using Release-like options for OptiX can be a workaround.

Thanks for letting me know the forum thread. Let me keep this issue opened until we find a proper way to fix.

— Reply to this email directly, view it on GitHub https://github.com/shocker-0x15/GfxExp/issues/6#issuecomment-1205563803, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB56PP67SGYKU4HGDMBM2GTVXP5H7ANCNFSM54JJKJYQ . You are receiving this because you authored the thread.Message ID: @.***>

-- -Steve http://berserko.hurleyworks.com/ http://www.hurleyworks.com

Hurleyworks avatar Oct 11 '22 08:10 Hurleyworks

I have been using OptiX-IR and Optix 7.6 and have not seen anymore problems. I also upgraded to a RTX 3090 :)

Been having lots of fun making renders with Image Based Lighting and this pathtracer http://berserko.hurleyworks.com/portfolio-items/cg/#

Hurleyworks avatar Jan 15 '23 01:01 Hurleyworks

Awesome work! I need to re-check the issue with the latest env (like OptiX 7.6-, CUDA 12.0) on my side as well.

shocker-0x15 avatar Jan 15 '23 18:01 shocker-0x15

Closing this issue.

shocker-0x15 avatar Dec 02 '23 19:12 shocker-0x15