agnaistic
"Generate More" appending character name
The "Generate More" button doesn't seem to work as it should. Say I get a reply that gets cut off mid-sentence because of the generation token limit.
{{user}}: Tell me about dinosaurs.
{{char}}: Oh, I really like dinosaurs! I liked them since I was a kid, ever since-
[.. insert a lot of babbling here ...]
-that's when I knew the Ambrosaurus would for
I want the AI to continue the response, so I click Generate More. But instead of simply continuing the response, it sends the following prompt:
{{user}}: Tell me about dinosaurs.
{{char}}: Oh, I really like dinosaurs! I liked them since I was a kid, ever since-
[.. insert a lot of babbling here ...]
-that's when I knew the Ambrosaurus would for
{{char}}: [insert continued generated text here]
Which very likely won't continue the text, instead starting a new reply and bundling them together in the UI. What should be sent instead is:
{{user}}: Tell me about dinosaurs.
{{char}}: Oh, I really like dinosaurs! I liked them since I was a kid, ever since-
[... insert a lot of babbling here ...]
-that's when I knew the Ambrosaurus would for[insert continued generated text here]
This is particularly difficult to fix. Removing the {{char}}: from the prompt doesn't guarantee that the AI will continue from the end of the last sentence. Each service effectively needs it's own solution.
I was looking into this last week for NAI, trying to insert a few differently phrased instructions to try to get the model to continue from where the previous message was interrupted as the same character, but results were unsatisfactory. NAI's instruct mode isn't as sophisticated as the dedicated chat models and doesn't seem to understand this very well, and without an instruction it really struggles to continue an interrupted message seamlessly when it sees the \n{{char}}: Agnai inserts.
The best results I've seen are to completely omit the \n{{char}}:, but sometimes this results in it wanting to switch to another chat participant which isn't ideal either.
Some sort of heuristic that could detect a mid-sentence "Generate More" (omit the \n{{char}}: in that case so that the model is very likely to finish that sentence and keep going) and an end-of-sentence "Generate More" (add the \n{{char}}: to discourage the model from switching to another character) seems like it would be optimal for NAI.
