infinity
infinity copied to clipboard
Disable model warmup
System Info
I am using 0.0.75 and when I use --no-model-warmup it still tries to warm up the model.
Therefor CTRL+C for exiting also does not work when warming up.
Maybe that only happens with ONNX. In PyTorch the --no-model-warmup does work.
Information
- [x] Docker + cli
- [x] pip + cli
- [ ] pip + usage of Python interface
Tasks
- [ ] An officially supported CLI command
- [ ] My own modifications
Reproduction
- Use
infinity_emb v2 --port 7997 --no-model-warmup --engine optimum --model-id <any-model> - Try to use or
curlto infinity but does not work because of ECONNREFUSED. - Try to CTRL+C but does not work. I need to wait until warm up is done.