onnxruntime_backend icon indicating copy to clipboard operation
onnxruntime_backend copied to clipboard

Published 20 hours ago • verified publisher icon triton-inference-server

Is onnxruntime-genai supported?

Open jackylu0124 opened this issue 1 year ago • 1 comments
trafficstars

Hey all, I have a quick question, is onnxruntime-genai (https://onnxruntime.ai/docs/genai/api/python.html) supported in Triton Inference Server's ONNX runtime backend? I couldn't find relevant sources in the documentation. Thanks in advance!

jackylu0124 avatar May 04 '24 15:05 jackylu0124

I came here for the same question! 😄

Ben-Epstein avatar May 09 '24 19:05 Ben-Epstein