Isaac Reis

https://www.linkedin.com/in/isaacisforreal/ [email protected]

Galadriel Porto, Portugal Senior Machine learning engineer

Results 4 comments of


                                            Isaac Reis

Are quantized models supported yet?

Any updates on this?

inference through text-embeddings-inference server is slower than inference directly using model

I'm having exactly the same issue in CPU. It takes more than 3x the time compared with the full sized model from HF (for 100 docs)

Slow inference time running on CPU

I'm having exactly the same issue in CPU with batch documents. 3X slower than with sentence_transformers. I'm using a c5.2xlarge from AWS

Add a way to identify or clean up AWS resources

Any updates on this?