exllama icon indicating copy to clipboard operation
exllama copied to clipboard

Published 20 hours ago • verified publisher icon turboderp

Continuous Batching support

Open FireMasterK opened this issue 1 year ago • 0 comments

vLLM, and HF's TGI can do this.

Additional Context: https://github.com/turboderp/exllama/issues/150#issuecomment-1633417028

FireMasterK avatar Aug 09 '23 00:08 FireMasterK