DeepRec
DeepRec copied to clipboard
Can deeprec processor be used in triton inference server?
We use triton inference server for online inference, Can deeprec processor be used in triton inference server?
Not supported yet. which feature do you want to use? TensorFlow/DeepRec saved_model can deploy in TritonServer deploy TensorFlow. If you wanna DeepRec feature, it supports. If you wanna SessionGroup, you can use TFServing or build RPC Server yourself.
We use triton for all model serving task(recommendation / nlp / cv), We want a triton deeprec backend with all the deeprec processor features for deeprec model serving: https://deeprec.readthedocs.io/en/latest/Processor.html @candyzone