FlexGen
FlexGen copied to clipboard
Soft Label of Flexgen
I tried to extract output prediction probs from pytorch_backend.py in flex_opt. But I can not get logits returns. I have little idea about extracting logits from your complex distribution computations. Thus, I am wondering if you could support the output of both ids and logits in apps?
Sure! I will work on this and upload the code soon.