pytorch_backend icon indicating copy to clipboard operation
pytorch_backend copied to clipboard

Published 20 hours ago • verified publisher icon triton-inference-server

Disable cudnn option

Open jamied157 opened this issue 1 year ago • 5 comments

We've occasionally had issues when multiple models that use cuDNN, (we sometimes see CUDNN_INTERNAL_ERROR and sometimes GPU memory will spike when running a kernel from cuDNN) so have found it beneficial to disable it in our own fork of the repo. It would be helpful to have an option to do this upstreamed.

If it would be helpful I could try and find a repro of the CUDNN_INTERNAL_ERROR issue but that may take a bit more time.

jamied157 avatar Jan 03 '24 10:01 jamied157

@Tabrizian are you okay to take a look at this? Thanks

jamied157 avatar Jan 09 '24 11:01 jamied157

Thanks for your contribution! Could you also add some documentation regarding this in the readme?

Done!

jamied157 avatar Jan 15 '24 18:01 jamied157

@Tabrizian is this being looked at?

HennerM avatar Feb 19 '24 20:02 HennerM

@kthui do you know who can review this change?

HennerM avatar Apr 12 '24 00:04 HennerM

@HennerM I'm so sorry I was out of the office when I was mentioned and might have missed the notification for this PR. This looks good to me. Thanks for your contribution. We need to run this PR through CI and add some testing before merging it. We'll merge the PR if the CI looks green.

Tabrizian avatar May 09 '24 17:05 Tabrizian

@Tabrizian Thanks for approving. Can you help with merging as well?

HennerM avatar May 17 '24 08:05 HennerM

@Tabrizian Thanks for approving. Can you help with merging as well?

Should be able to merge it soon. Sorry for the delay.

Tabrizian avatar Jun 01 '24 03:06 Tabrizian