Cohee

Let me provide more context as I rely on Textgen API usage in my project (SillyTavern). I think the problem is that adding a new parameter to the payload that...

`/api/textgen` has been deleted and merged with 5000/api See: #990

Actually, never mind the extensions part. I took a closer look at the console logs and noticed that it may be related to the LLaMa tokenizer producing an extra leading...

https://huggingface.co/docs/transformers/main/model_doc/llama#:~:text=The%20LLaMA%20tokenizer,the%20tokenizer%20configuration. Looks like the case. decode_with_prefix_space exists in the code but supposedly does nothing right now. A dumb quick solution would be to pad the "original_question" string with a leading...

@oobabooga Maybe it's worth trying decoding only the newly generated tokens instead of the whole batch. Consider the following code: ``` original_tokens_count = len(input_ids[0]) generated_tokens_count = len(output) new_tokens = generated_tokens_count...

> Thanks a lot @SillyLossy, that indeed seems to fix it. I have incorporated the changes here: https://github.com/oobabooga/text-generation-webui/commit/de6a09dc7f7d5a5d8496cfa1598abb4ff5ee1338 That indeed worked perfectly. Did some testing with llama again and I...

`{{idle_duration}}` macro added to the dev branch

"Idle" extension can be downloaded using a built-in extensions downloader.

Added saving presets based on current settings to dev branch. https://github.com/Cohee1207/SillyTavern/commit/e3b9c24d8c4650f8907fb3cb97ced23b0161ad12 For prompt boxes, I can suggest increasing the default height to 6 rows and making lines there a little...

I believe this was fixed some time ago.