onnxruntime_backend icon indicating copy to clipboard operation
onnxruntime_backend copied to clipboard

Update onnxruntime to 1.14.0 or 1.14.1 to fix TensorRT issue

Open OvervCW opened this issue 2 years ago • 0 comments
trafficstars

Is your feature request related to a problem? Please describe. We are currently unable to (properly) use TensorRT in Triton because there is a bug in the onnxruntime < 1.14.0 that makes TensorRT engine caching unreliable. This bug has been fixed in version 1.14.0, so it would be nice if Triton's backend could be updated to that version.

Describe the solution you'd like Update the backend to use onnxruntime 1.14.0 or later.

Describe alternatives you've considered To not use TensorRT for now.

Additional context N/A

OvervCW avatar Mar 10 '23 16:03 OvervCW