serving icon indicating copy to clipboard operation
serving copied to clipboard
verified publisher icon tensorflow

CUDA Graphs support for Tensorflow Serving

Open ndeep27 opened this issue 1 year ago • 2 comments

Does TF Serving support CUDA graphs?

ndeep27 avatar Apr 09 '24 04:04 ndeep27

@singhniraj08 Can you please help with above query? I want to see if there is a way to convert a TF graph to CUDA graph for serving optimization

ndeep27 avatar Apr 26 '24 22:04 ndeep27

@ndeep27, I don't think Tensorflow serving supports CUDA graphs. We will keep this as a feature request and and I will pass this feature to owners internally for implementation. Thank you!

singhniraj08 avatar Apr 29 '24 05:04 singhniraj08