Bainainai

Some of the client exposed features of web-llm require tokenization and decoding of tokens to be used effectively. The tokenizer is already loaded for web-llm's internal functionality and can be...

If you close a page window while that page is streaming results from the service worker, the service worker becomes permanently unable to process any further results. To fix this,...

Is there any way to save/restore the intermediary inference state or do any kind of prompt caching?