gpt4all
gpt4all copied to clipboard
nomic-ai
🤔🧠Persisting Context Across Prompts
One of the must have features on any chatbot is conversation awareness. You say your name and it remembers, so the context is stored among prompts.
I believe context should be something natively enabled by default on GPT4All.
After some research I found out there are many ways to achieve context storage, I have included above an integration of gpt4all using Langchain (I have converted the model to ggml. The code is adapted to GPT4All from this Langchain example about ConversationChain and ConversationSummaryMemory to create summarization of context between the outputs (the other way would be to include the whole conversation on the input, what would quickly hit the tokens limit).
Note that this may take a bit longer to generate/read the summaries.
from langchain.llms import GPT4All
from langchain.chains import ConversationChain
from langchain.chains.conversation.memory import ConversationSummaryMemory
from langchain import PromptTemplate
# Initialize the GPT model
llm = GPT4All(model="./gpt4all-lora-unfiltered-quantized.bin")
# Define the prompt template for the ConversationChain
template = """Current conversation:
{history}
Human: {input}
AI:"""
prompt = PromptTemplate(template=template, input_variables=["input", "history"])
# Initialize the ConversationChain with ConversationSummaryMemory
conversation = ConversationChain(
llm=llm,
memory=ConversationSummaryMemory(llm=llm),
prompt=prompt,
)
# Start an infinite conversation loop
while True:
# Prompt the user for input
user_input = input("You: ")
# Generate a response from AI based on the user input and conversation history
response = conversation.__call__(user_input, conversation.memory.buffer)
# Print AI's response
print("AI:", response)
Although this code works, I'm looking forward to know what are other better ways to achieve conversation context for gpt4all!
I've been pondering skewing embeddings off of added context?
But I'd seen a guy just read from and modify a text file over on chatGPTs discord for continuous recall.
How did you convert the model? All I get is "llama_model_load: invalid model file './gpt4all-lora-unfiltered-quantized.bin' (too old, regenerate your model files or convert them with convert-unversioned-ggml-to-ggml.py!)", but trying to do that just gets me a different error...
How did you convert the model? All I get is "llama_model_load: invalid model file './gpt4all-lora-unfiltered-quantized.bin' (too old, regenerate your model files or convert them with convert-unversioned-ggml-to-ggml.py!)", but trying to do that just gets me a different error...
#215
One of the must have features on any chatbot is conversation awareness. You say your name and it remembers, so the context is stored among prompts.
I believe context should be something natively enabled by default on GPT4All.
After some research I found out there are many ways to achieve context storage, I have included above an integration of gpt4all using Langchain (I have converted the model to ggml. The code is adapted to GPT4All from this Langchain example about ConversationChain and ConversationSummaryMemory to create summarization of context between the outputs (the other way would be to include the whole conversation on the input, what would quickly hit the tokens limit).
Note that this may take a bit longer to generate/read the summaries.
from langchain.llms import GPT4All from langchain.chains import ConversationChain from langchain.chains.conversation.memory import ConversationSummaryMemory from langchain import PromptTemplate # Initialize the GPT model llm = GPT4All(model="./gpt4all-lora-unfiltered-quantized.bin") # Define the prompt template for the ConversationChain template = """Current conversation: {history} Human: {input} AI:""" prompt = PromptTemplate(template=template, input_variables=["input", "history"]) # Initialize the ConversationChain with ConversationSummaryMemory conversation = ConversationChain( llm=llm, memory=ConversationSummaryMemory(llm=llm), prompt=prompt, ) # Start an infinite conversation loop while True: # Prompt the user for input user_input = input("You: ") # Generate a response from AI based on the user input and conversation history response = conversation.__call__(user_input, conversation.memory.buffer) # Print AI's response print("AI:", response)Although this code works, I'm looking forward to know what are other better ways to achieve conversation context for gpt4all!
Jeen-Yuhs! Thank you so much !
