infinity icon indicating copy to clipboard operation
infinity copied to clipboard

Disable model warmup

Open isikemre opened this issue 10 months ago • 0 comments

System Info

I am using 0.0.75 and when I use --no-model-warmup it still tries to warm up the model.
Therefor CTRL+C for exiting also does not work when warming up.

Maybe that only happens with ONNX. In PyTorch the --no-model-warmup does work.

Information

  • [x] Docker + cli
  • [x] pip + cli
  • [ ] pip + usage of Python interface

Tasks

  • [ ] An officially supported CLI command
  • [ ] My own modifications

Reproduction

  1. Use infinity_emb v2 --port 7997 --no-model-warmup --engine optimum --model-id <any-model>
  2. Try to use or curl to infinity but does not work because of ECONNREFUSED.
  3. Try to CTRL+C but does not work. I need to wait until warm up is done.

isikemre avatar Feb 28 '25 16:02 isikemre