DeepSpeed-MII icon indicating copy to clipboard operation
DeepSpeed-MII copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Support for token streaming

Open Archmilio opened this issue 8 months ago • 0 comments

Thank you for your hard work. I am really excited about MII performance.

I have some questions

Does token streaming function supported now?

If token streaming is supported, I would like to test the first token latency and completion time. Do you happen to know when it will be supported?

Archmilio avatar Nov 15 '23 08:11 Archmilio