clip-as-service icon indicating copy to clipboard operation
clip-as-service copied to clipboard

Published 20 hours ago • verified publisher icon jina-ai

Onyx + TensorRT support for XLM-R Large Vit-B/16+?

Open NOT-HAL9000 opened this issue 2 years ago • 3 comments

Just wondering if this is possible? Thanks!

NOT-HAL9000 avatar Sep 15 '22 00:09 NOT-HAL9000

Hi, thank you for asking!

For ONNX it is possible, and it will be supported in future releases. Unfortunately, we have not yet planned to support TensorRT for M-CLIP models very soon. You may refer to other online resources on converting PyTorch models to TensorRT.

ZiniuYu avatar Sep 15 '22 09:09 ZiniuYu

For ONNX it is possible, and it will be supported in future releases

Thanks! Is there a roadmap with a rough ETA for the ONYX runtime for M-CLIP models?

NOT-HAL9000 avatar Sep 15 '22 11:09 NOT-HAL9000

It's hard to offer the exact timeline for supporting ONNX M-CLIP models. However, It's hopefully we can release it by end of this month.

numb3r3 avatar Sep 19 '22 06:09 numb3r3

@NOT-HAL9000 I'm glad to say now, ONNX M-CLIP models have been enabled in the latest version. Please play it around, and leave your comments.

numb3r3 avatar Sep 27 '22 02:09 numb3r3

@numb3r3 Thank you!

NOT-HAL9000 avatar Oct 13 '22 00:10 NOT-HAL9000