opencompass icon indicating copy to clipboard operation
opencompass copied to clipboard

Published 20 hours ago • verified publisher icon open-compass

[Feature] Improve the inference speech with vLLM batch API

Open tonysy opened this issue 7 months ago • 0 comments

Describe the feature

Improve the inference speech with vLLM batch API

Will you implement it?

  • [ ] I would like to implement this feature and create a PR!

tonysy avatar Jul 19 '24 14:07 tonysy