nm-vllm
nm-vllm copied to clipboard
[WIP] afeldman-nm/encoder decoder
GOALS • Whisper support • Exemplifies encoder/decoder (E/D) support • E/D K/V caching • E/D parallelism
TESTING • HuggingFace whisper model • Replicate public English Speech Recognition (SR) test using canned audio • Audio front-end is out-of-scope of this PR