How to avoid suboptimal performance when using multiple GPU

Open mingxuancai opened this issue 1 year ago • 1 comments

Hi,

My server has 3 different GPUs (A6000/Titan Xp/3090). I successfully installed tinycudann, but when I import tinycudann, it has the following warning:

"UserWarning: System has multiple GPUs with different compute capabilities: [86, 61, 86]. Using compute capability 61 for best compatibility. This may result in suboptimal performance."

I tried to only use A6000 for Fully fused MLP, but it tells me it still degraded to CUTLASS MLP. Is there any way to avoid the suboptimal performance?

Thank you!

Dec 06 '24 20:12 mingxuancai

"UserWarning: System has multiple GPUs with different compute capabilities: [86, 61, 86]. Using compute capability 61 for best compatibility. This may result in suboptimal performance."

"tiny-cuda-nn warning: FullyFusedMLP is not supported for the selected architecture 61. Falling back to CutlassMLP. For maximum performance, raise the target GPU architecture to 75+."

From my experience, if there is one of the GPUs below 75, then the tinycudann just directly uses the lowest capability. However, even if I choose to use the 75+ GPU, it cannot change the situation. So how can I raise the target GPU architecture to 75+?

Dec 07 '24 06:12 mingxuancai