Handle different Histories in same session / Use Custom History

[LSXAxeller]

I'm setting up various inferences backends in my project, including LLamaSharp. I wanted to keep a unified history across these backends, so I created a custom structure to handle it. However, I couldn't use ChatSession with this custom history. So, I decided to directly use InteractiveExecutor for inferences.

Since my project allows editing and regenerating user messages similar to ChatGPT, I needed to use a custom history to switch between histories. But I noticed that all sent text gets tokenized from the beginning as one history in _embed_inps. This means that using a custom history is pointless since I can't swap between different histories.

My question is, is there a method to swap the history in the executor?

[philippjbauer]

Is using the save/load functionality not working for you? It saves the model state and a json file for your history into a customizable folder. If this is not what you're looking for, can you explain what you are trying to do in more detail? I'm not quite understanding how your unified history is working and what might be incompatible.

https://github.com/SciSharp/LLamaSharp/blob/master/LLama/ChatSession.cs#L113

[LSXAxeller]

Is using the save/load functionality not working for you? It saves the model state and a json file for your history into a customizable folder. If this is not what you're looking for, can you explain what you are trying to do in more detail? I'm not quite understanding how your unified history is working and what might be incompatible.

https://github.com/SciSharp/LLamaSharp/blob/master/LLama/ChatSession.cs#L113

ChatGPT allows user to edit messages then AI regenerate it with different history, now I found that ChatSession is just for Show/Keeping track of messages, but doesn't affect the executor history, since executor remembers history as token list in '_embed_inps' and I can't edit this history and swap it like user would on message edit, since that means I need to recreate the executor and fill the history each time I press swap message button

like this, the four messages is just the same message but edited and each response doesn't remember it's previous, and keep track with edited history