FastChat
FastChat copied to clipboard
update vllm_worker.py
Why are these changes needed?
- The vllm_worker code does not catch exceptions: when I check vllm's SamplingParams function, there is parameter verification and the error ValueError is thrown out.
- Because there is no capture, the semaphore is not released after the code execution fails, causing the model to fail to work.
Related issue number (if applicable)
Fix #3328
Checks
- [ ✓] I've run
format.shto lint the changes in this PR. - [ ✗] I've included any doc changes needed.
- [ ✓] I've made sure the relevant tests are passing (if applicable).