Huadong Liao

icon indicating a website link www.linkedin.com/in/naturomics icon indicating an email [email protected]

icon location Chongqing, China icon location Explore Nature from an Omics Perspective

Results 1 issues of Huadong Liao

Speculative decoding leads to zombie requests

### Anything you want to discuss about vllm. when turns on speculative decoding, the openai api server failed on concurrent requests, raising error `exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup`. After...

misc