onnxruntime_backend icon indicating copy to clipboard operation
onnxruntime_backend copied to clipboard

Published 20 hours ago • verified publisher icon triton-inference-server

Add option to enable CUDA Graphs in CUDA EP

Open nealvaidya opened this issue 2 years ago • 0 comments
trafficstars

ONNXRuntime has added support in "preview mode" for CUDA Graphs in the CUDA Execution Provider. It would be useful to expose this option for the onnx runtime Triton backend as well to help reduce CPU Utilization.

nealvaidya avatar Feb 15 '23 23:02 nealvaidya