add batching in ask response stream

Open yujonglee opened this issue 1 year ago • 1 comments

Aug 21 '24 02:08 yujonglee

currently we are using model that is "fast enough", so not needed at the moment

Aug 21 '24 04:08 yujonglee