[Feature Request] History

[cachho]

On discord, this was discussed as part of #105, but I think it should be tracked separately.

History can greatly improve the quality of the bot. Without history, queries like tell me more can't be processed.

Me: how do I write a hello world program in python?
AI: To write a "Hello, World!" program in Python, you can use the following code: print("Hello, World!") This will display the phrase "Hello, World!" on the console when you run the program.
Me: tell me more
AI: I'm sorry, but without any specific context or information about the subjects you are referring to, I cannot provide any further details.

You can deal with this by appending previous questions and answers to each prompt. The obvious down side of this is that the costs are multiplied really quick.

Implementation

I'm pretty sure you can just add it to the prompt, something like: You are aware of this history: ... The cut off could be a number of messages or a token count.

The biggest question from an implementation standpoint is, how much of this should be handled server side, how much client side?. (Considering the (usual) scenario where embedchain is implemented as an API and a chatbot frontend sends requests with querries.)

I know we're trying to abstract here, so handle as much on the server as possible. But the problem with that is, that there might be multiple users (sessions). And you don't want the history of someone else. Of course you could store the history on the server with a session id that's provided by the client to solve this problem. Then you would still end up with more boilerplate (session id generation and passing), and you would have to come up with a system to clear sessions due to memory/storage limitations.

That's why I think it should be handled on the client. Most chat frontends will have a messages array anyways. Then you can just pass that with the request and embedchain can handle it as a optional parameter and append it to the prompt. Yes, it's more stuff that the user has to handle, but it's safer for the memory.

What do you think?