tutorials icon indicating copy to clipboard operation
tutorials copied to clipboard

Published 20 hours ago • verified publisher icon triton-inference-server

Example usage of kafka endpoints as an input and output for triton server using Python API

Open pthalasta opened this issue 9 months ago • 0 comments

This PR provides an example of using kafka endpoints as an input and output for triton server using Python API. As an example code, we have deployed a preprocessing stage of tokenizer as a custom model.

Input: Each message in the kafka topic Output: json string of the output produced after passing the input through the deployed model

This PR has an example to support #3131 using triton's python API

pthalasta avatar May 09 '24 03:05 pthalasta