Cade Truitt
Results
2
comments of
Cade Truitt
This is a known issue with the current implementation of the existing failure backend. Currently, failures are removed based on a positional index rather than a unique id, so if...
As much as I would _love_ to take credit for bringing Speculative Decoding to vLLM, I'm relatively certain the praise belongs to @cadedaniel. 😁