FastChat
FastChat copied to clipboard
Integrate vllm Error, TypeError: top_k must be an integer, got float
trafficstars
When I use fastchat to integrate vllm, I get the error "TypeError: top_k must be an integer, got float". The reason is that vllm 0.5.5 has added a BUGFIX #7227, check top_k must be an integer。For fastchat, the default value of top_k in vllm_worker.py is set to -1.0.