text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

feat(server): pre-allocate past key values for flash causal LM

Open OlivierDehaene opened this issue 2 years ago • 0 comments
trafficstars

OlivierDehaene avatar Jun 05 '23 15:06 OlivierDehaene