The bot breaks into talking nonsense after some messages

[OcelotTheOcelot]

After a number of messages (depends on their token volume), the bot starts to talk gibberish like a mental asylum patient (beam search and temperature are at their default value: 1). I encounter this issue both in the online demo and on my local machine (7B based, running on Windows) Is there a way to make it keep the user's first message in memory all the time and shift attention for the following ones like they are in a queue? Or maybe a workaround for this issue, for example, a way to save the current state of the conversation and continue it after restarting the bot.