litgpt icon indicating copy to clipboard operation
litgpt copied to clipboard

Published 20 hours ago • verified publisher icon Lightning-AI

Stream option

Open rasbt opened this issue 9 months ago • 3 comments

Add a stream=True option to generate

rasbt avatar May 14 '24 12:05 rasbt

Hey @rasbt, are you looking for contributions to this enhancement? If so, can you share your ideas on where and how to start with streaming so I can contribute?

sairampillai avatar May 15 '24 16:05 sairampillai

Thanks for your interest in this. This was requested by a colleague for a downstream application, but I haven't had a chance to think this through in terms of how this would look like (am currently traveling and haven't had a chance to follow up). I added it as a note for now but will probably have to implement it in tandem with the application.

rasbt avatar May 16 '24 11:05 rasbt

@rasbt Got it, thank you.

sairampillai avatar May 17 '24 13:05 sairampillai