rust-bert
rust-bert copied to clipboard
Question: is it ok to continue after OOM error from `encode`
I run a sentence-transformer in a http service to serve requests. And I've ran into OOM a few times during encode and noticed that it returns an error instead of panicing.
My experience in Java tells me that it's difficult to recover from OOM so the service may as well just crash. But this lib has chosen to return an error, I wonder if it has taken precausion (e.g. to check available memory before allocating), hence my silly question: is it ok to use the model instance to encode again after OOM in the process.