instant-ngp icon indicating copy to clipboard operation
instant-ngp copied to clipboard

Error "cutlass_matmul.h:332 status failed with error Error Internal" on freshly compiled Instant-ngp

Open MattRM2 opened this issue 1 year ago • 7 comments

Hi,

I just compiled Instant-ngp and I have this error when I tried to load the Fox exemple (see screenshot). I had the same with the build from NVlabs but know it's compiled for my system.

Instant-ngp_Error

Do you have a solution ? Please, a real solution not just deactivated all my card from my system but the GTX 970. All my system is detected correctly, so please not a fix like "turn off all the graphics cards and keep only one". This is not a solution, it's a bad workaround.

Thank you,

Matt

MattRM2 avatar Jun 01 '23 16:06 MattRM2

same issue here

david9ml avatar Jun 14 '23 08:06 david9ml

same issue!

weizizhuang avatar Jun 15 '23 04:06 weizizhuang

did someone figure it?

LudovicoYIN avatar Jul 06 '23 05:07 LudovicoYIN

Right now, no.

MattRM2 avatar Jul 06 '23 06:07 MattRM2

Fwiw, I got the same error ("cutlass_matmul.h:332 ...") with the binary built at commit 090aed613499ac2dbba3c2cede24befa248ece8a, but building at commit 6ee7724370b5e7aa9d514a8c36fae78c112706d9 and dragging a source file (a directory or a transforms.json) into the GUI window works properly. The weird thing is that using the command line to specify the scene, e.g. ./instant-ngp --scene data/nerf/fox still gives the above error, the error can only be avoided if I drag the source into the GUI window.

blurgyy avatar Jul 17 '23 14:07 blurgyy

This is related to your machine GPU and train_ batch_ size, GPU70 (maybe) and below architectures prevent the use of Fully Fused MLP and only cutlassMLP can be used, while the train_ batch_ size is 18+, cutlassMLP will report this error! @david9ml @MattRM2 @weizizhuang @blurgyy @LudovicoYIN

jx-99321 avatar Jan 26 '24 07:01 jx-99321

my settings is GPU 4060Ti (89 architecture),training_batch_size=21.

jx-99321 avatar Jan 26 '24 07:01 jx-99321