llm-ui icon indicating copy to clipboard operation
llm-ui copied to clipboard

Published 20 hours ago • verified publisher icon llm-ui-kit

How to use throttle function to speed up the output display

Open tqdo opened this issue 1 year ago • 2 comments

Throttle function has 5 parameters and I have no idea how to use them to speed up the display. Can we just combine them into 1 variable like the slide bar you have in your doc? It is so much more intuitive. https://llm-ui.com/docs/quick-start

tqdo avatar Oct 06 '24 18:10 tqdo

to this same point: can we add a "fastForward" option or something? A lotta times if the backend is too slow, llm-ui overcompensates and does very very slow output. it would be good to periodically be able to fast-forward to the latest output manually.

danielrussellLA avatar Apr 14 '25 22:04 danielrussellLA

More wish list items for throttle:

  • A callback to know when the internal buffering of the throttle is completed and it actually starts showing something. I have a loading bubble and it is visible that it disappears before llm-ui starts printing.

  • A way to slow things down if the backend is really fast (e.g. gemini-flash fast). It just is stressful for readers if the text just comes out too fast.

coezbek avatar Jun 25 '25 22:06 coezbek