AI starts chatting with itself after a few questions.

[daveckw]

trafficstars

Describe the bug

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Just ask a few questions continuously, after that it will start chatting with itself as shown in the sceenshot.

Screenshot

No response

Logs

No error in the command line.

System Info

Self hosted. GPU: RTX3090. AMD Ryzen Processor 5950X

[Originalimoc]

Looks like vicuna prompt

[Hiriyuki]

sounds like a feature than a bug haha

[elcotek]

The same for me, it is independent of the GPU Vram use. After a while, the Cai Bot starts to lose his context to the actual requests. Also, it seems like he's no longer going through the real log. This is easy to recognize, because the GPU memory consumption drops after a while.

I'm just getting started with Phython, I'm from the C# world

[daveckw]

Looks like vicuna prompt

Yes, Vicuna checkpoint

[mcmonkey4eva]

See my answer explaining this @ https://github.com/oobabooga/text-generation-webui/discussions/839#discussioncomment-5557486

[Enferlain]

Same behavior in every single ggml model so far for me, both on oobabooga webui and koboldcpp. On ooba webui you can easily eliminate this with "stop generation on new line" but on koboldcpp there is no option like this, so it's a painful experience.

[ltngonnguyen]

This is fixed with the latest version of Text Generation WebUI, just do a git pull, and go to Parameters -> Custom Stop Strings -> add "Human:", "User:", "Assitant:" or whatever the start of the hallucination is. Check out my guide here for more details: https://github.com/ltngonnguyen/vicuna-matata/

[github-actions[bot]]

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.