Serving SeTFiT models in torchserve

[maccarini]

Im trying to deploy my setfit model in torchserver using a custom handler for this task. The thing is that im not being able to do this since im getting multiple errors while registering the model on the service. I was wondering if this is actually possible since i tried to serve sentence-transformers models before successfully. Also to clarify, Im including setfit in requirements.txt

[lsmith77]

would also be interested to learn about this.

[puppetm4st3r]

@maccarini can you resolve it? i'm looking a way to serv the model, I can temporarily add this models to my embeddings and re-ranking server repo but it is not a solution that makes the most of GPU resources usage, however it is a good starting point if we have no other option (for now)