cudaDeviceSynchronize() failed with error operation not permitted when stream is capturing

[seaprogrammer]

Anyone know what this is? I receive the message and the software returns me to command line if I have aabb_scale set higher than 2. System: Debian Linux. instant-ngp built from git a few days ago.

22:45:02 INFO     Loading NeRF dataset from
22:45:02 INFO       data/pulseox/transforms.json
22:45:02 SUCCESS  Loaded 98 images after 0s
22:45:02 INFO       cam_aabb=[min=[-0.444531,-1.33551,-1.07948], max=[3.62585,2.37042,4.1108]]
22:45:03 INFO     Loading network config from: configs/nerf/base.json
22:45:03 INFO     GridEncoding:  Nmin=16 b=1.7411 F=2 T=2^19 L=16
Warning: FullyFusedMLP is not supported for the selected architecture 37. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
Warning: FullyFusedMLP is not supported for the selected architecture 37. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
Warning: FullyFusedMLP is not supported for the selected architecture 37. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+.
22:45:03 INFO     Density model: 3--[HashGrid]-->32--[FullyFusedMLP(neurons=64,layers=3)]-->1
22:45:03 INFO     Color model:   3--[Composite]-->16+16--[FullyFusedMLP(neurons=64,layers=4)]-->3
22:45:03 INFO       total_encoding_params=13918896 total_network_params=9728
22:45:03 WARNING  Vulkan error: loaderValidateLayers: Layer 0 does not exist in the list of available layers
22:45:03 WARNING  Could not initialize Vulkan and NGX. DLSS not supported. (/home/jag/opt/src/nerf/mesh/instant-ngp/src/dlss.cu:220 vkCreateInstance(&instance_create_info, nullptr, &vk_instance) failed)
Got cutlass error: Error Internal at: 346
Could not free memory: /home/jagga/pub/src/instant-ngp/dependencies/tiny-cuda-nn/include/tiny-cuda-nn/gpu_memory.h:465 cudaDeviceSynchronize() failed with error operation not permitted when stream is capturing

[fpibbs]

Same issue here: windows 10.

[sumanttyagi]

any solution? @seaprogrammer