text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

[Prototype] Vectorized causal lm

Open jlamypoirier opened this issue 2 years ago • 0 comments

Prototype to greatly reduce the post-processing overhead at higher batch sizes.

jlamypoirier avatar May 03 '23 15:05 jlamypoirier