mouchourider

> You can see a description of how the API works [here](https://github.com/oobabooga/text-generation-webui/blob/main/api-example.py#L20) > > The character's identity is given by the prompt, and the prompt is basically a concatenation of...

You're a saint. Thanks

> Edit: solved by basically re-creating what the chat ui was doing. formatting the input prompt with a brief at least 5+ message long history of messages separated by newlines...

> The problem remaining with the above is that the API seems to IGNORE stopping_strings so it does NOT stop generating and it just hallucinates a bunch of simulated conversation...

> @DeSinc @mouchourider, did you ever figure out stopping strings? Nope. Still stuck on this.

You need to change the requirement.txt file to this: datasets loralib sentencepiece git+https://github.com/zphang/transformers@c3dc391 accelerate bitsandbytes git+https://github.com/huggingface/peft.git gradio And you need to change those three functions as follow in the main.py...

> https://github.com/tloen/alpaca-lora with a few changes seem to be a good option. (Just by changing the generate_prompt to something more chat conversation oriented than instructions oriented). > > https://github.com/deep-diver/Alpaca-LoRA-Serve is...

> Is there a way to keep the `Signed Entities` enabled and pass the key to the component? I would really like to know about that either.