lorax icon indicating copy to clipboard operation
lorax copied to clipboard

Published 20 hours ago • verified publisher icon predibase

Fp6 quant from deepspeed

Open flozi00 opened this issue 1 year ago • 2 comments
trafficstars

Feature request

https://github.com/huggingface/text-generation-inference/issues/1633

Motivation

Throughout and latency

Your contribution

@tgaddair what do you think?

flozi00 avatar Mar 08 '24 16:03 flozi00

Hey @flozi00, looks promising! Do you have bandwidth to open a PR for this?

tgaddair avatar Mar 10 '24 20:03 tgaddair

Yes, probably middle or end of the week

flozi00 avatar Mar 10 '24 20:03 flozi00