Cohee

Use the author's note with scenario positioning and set a default value for the author's note to your liking. Assuming you're not already using author's notes

Released.

From the repo readme: > The Python API of mlc_llm.MLCEngine fully aligns with OpenAI API You can use it as a Custom Chat Completion endpoint then. No need for a...

Your settings look okay. But I think you should try without multigen, it tends to create repeated phrases.

You did not specify the branch you're using, please use the bug template next time. I assume this relates to tiktoken tokenizer being instantiated for every tokenization request. I tested...

Just a quick question, maybe perhaps you have a monitor with high refresh rate (160Hz or more?)

@hmage thanks a lot for the helpful insights. I was able to add a couple of possible improvements to address this: 1. Tokenization of OAI messages doesn't block the main...

@rsmyst please check if it's any better for you after the recent optimizations. These are in the dev branch.

Added caching of token counts to release branch for all APIs and tokenizers, including LLaMA. Please reopen if that persists.

The creator of 11labs extension pulled Silero support for it (would require extras server running). If you are not happy enough with that, we could start working on Bark /...