[Feature] Prevent system messages from being pushed out of context

[nullstreak]

trafficstars

Problem Description The system messages (such as instructions, prompts, or rules) stay at the bottom of the messages stack. As the conversation context gets longer, the system messages are pushed out of the context and eventually stop working.

Proposed Solution I suggest adding a toggle option that automatically moves the last system message to the second to last position at the messages array (in the backend, keep the interface unchanged), just before the last user's message. This way, the instructions are always have high priority in the context of the LLM.

[Bin-Huang]

That's pretty much how it works. The system message always comes first in the context.

[nullstreak]

@Bin-Huang This is clearly not the case in version 1.2.6.

[nullstreak]

@Bin-Huang Same in the version 1.3.0.

[Bin-Huang]

This confuses me. Isn't the example you provided proving that everything is normal? The first entry in the content field is a system prompt, and it remains the first entry as the context grows.

[nullstreak]

@Bin-Huang That's right. But entries that are closer to the end have a higher priority, so it should be something like this:

• User message 1
• Assistant message 1
• User message 2
• Assistant message 2
• System message
• User message 3

In this case, the latest user message has the highest priority. After that, the system message has the second-highest priority, and then comes the rest of the context.

On the contrary, the instruction that remains at the beginning slowly loses its power as the context grows. And it doesn't take that long. After 3-4 messages, the system prompt wouldn't be followed as closely as when it started.

[Bin-Huang]

I'm sorry for only understanding what you've meant till now. I think you are right, and perhaps we should add a toggle option for this feature.