lorax icon indicating copy to clipboard operation
lorax copied to clipboard

Published 20 hours ago • verified publisher icon predibase

FlashInfer integration + cascade inference (prefix caching)

Open tgaddair opened this issue 1 year ago • 1 comments
trafficstars

See https://flashinfer.ai/2024/01/08/cascade-inference.html

tgaddair avatar Feb 05 '24 17:02 tgaddair

https://x.com/ye_combinator/status/1754537687422497220?s=20

Another nice source about that I think this could be a high priority feature ?

flozi00 avatar Feb 07 '24 15:02 flozi00