rust-bert icon indicating copy to clipboard operation
rust-bert copied to clipboard

Question: is it ok to continue after OOM error from `encode`

Open JackKCWong opened this issue 1 year ago • 0 comments

I run a sentence-transformer in a http service to serve requests. And I've ran into OOM a few times during encode and noticed that it returns an error instead of panicing. My experience in Java tells me that it's difficult to recover from OOM so the service may as well just crash. But this lib has chosen to return an error, I wonder if it has taken precausion (e.g. to check available memory before allocating), hence my silly question: is it ok to use the model instance to encode again after OOM in the process.

JackKCWong avatar Feb 05 '24 13:02 JackKCWong