DeepRec icon indicating copy to clipboard operation
DeepRec copied to clipboard

Can deeprec processor be used in triton inference server?

Open supercocoa opened this issue 1 year ago • 2 comments

We use triton inference server for online inference, Can deeprec processor be used in triton inference server?

supercocoa avatar Nov 01 '23 04:11 supercocoa

Not supported yet. which feature do you want to use? TensorFlow/DeepRec saved_model can deploy in TritonServer deploy TensorFlow. If you wanna DeepRec feature, it supports. If you wanna SessionGroup, you can use TFServing or build RPC Server yourself.

candyzone avatar Nov 09 '23 03:11 candyzone

We use triton for all model serving task(recommendation / nlp / cv), We want a triton deeprec backend with all the deeprec processor features for deeprec model serving: https://deeprec.readthedocs.io/en/latest/Processor.html @candyzone

supercocoa avatar Nov 12 '23 01:11 supercocoa