GPT4All and create_csv_agent : llama_generate: error: prompt is too long (680 tokens, max 508)

[Tukks]

I don't want to use ChatGpt, I want to use a local "chatgpt" So I tried this example with GPT4All https://python.langchain.com/en/latest/modules/agents/toolkits/examples/csv.html

from langchain.llms import GPT4All
from langchain import PromptTemplate, LLMChain
from langchain.agents import create_csv_agent
import pandas as pd

df = pd.read_csv('./titles.csv')

model = GPT4All(model="./models/ggml-alpaca-7b-q4-new.bin")
agent = create_csv_agent(model, './titles.csv', verbose=True)
>>> agent.run("how many rows are there?")

I have the error : llama_generate: error: prompt is too long (680 tokens, max 508)

Tried with model = GPT4All(model="./models/ggml-alpaca-7b-q4-new.bin") and model = GPT4All(model="C:\\Users\\glapenta\\Workspace\\python\\models\\ggml-alpaca-7b-q4-new.bin", n_ctx=1024, n_threads=8) I have this error with n_ctx at 1024 llama_generate: error: prompt is too long (1056 tokens, max 1020)

Here's the full stack :

>>> agent.run("how many rows are there?")


> Entering new AgentExecutor chain...
llama_generate: seed = 1680965712

system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
sampling: temp = 0.800000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000
generate: n_ctx = 512, n_batch = 1, n_predict = 256, n_keep = 0


 [end of text]

llama_print_timings:        load time = 74735.64 ms
llama_print_timings:      sample time =    51.08 ms /   116 runs   (    0.44 ms per run)
llama_print_timings: prompt eval time = 136599.33 ms /   514 tokens (  265.76 ms per token)
llama_print_timings:        eval time = 854042.23 ms /  1599 runs   (  534.11 ms per run)
llama_print_timings:       total time = 61952108.58 ms

You are working with a pandas dataframe in Python. The name of the dataframe is `df`.
You should use the tools below to answer the question posed of you:

python_repl_ast: A Python shell. Use this to execute python commands. Input should be a valid python command. When using this tool, sometimes output is abbreviated - make sure it does not look abbreviated before using it in your answer.

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [python_repl_ast]
Action Input: the input to the action
Observation: the action to take, should be one of [python_repl_ast] is not a valid tool, try another one.
Thought:llama_generate: seed = 1680965951

system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
llama_generate: error: prompt is too long (678 tokens, max 508)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\chains\base.py", line 213, in run
    return self(args[0])[self.output_keys[0]]
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\chains\base.py", line 116, in __call__
    raise e
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\chains\base.py", line 113, in __call__
    outputs = self._call(inputs)
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\agents\agent.py", line 790, in _call
    next_step_output = self._take_next_step(
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\agents\agent.py", line 679, in _take_next_step
    output = self.agent.plan(intermediate_steps, **inputs)
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\agents\agent.py", line 398, in plan
    action = self._get_next_action(full_inputs)
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\agents\agent.py", line 360, in _get_next_action
    parsed_output = self._extract_tool_and_input(full_output)
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\agents\mrkl\base.py", line 140, in _extract_tool_and_input
    return get_action_and_input(text)
  File "C:\Users\glapenta\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\langchain\agents\mrkl\base.py", line 48, in get_action_and_input
    raise ValueError(f"Could not parse LLM output: `{llm_output}`")
ValueError: Could not parse LLM output: ``

Great framework !

[cxfvic]

Same issue here

[TestCode2080]

Think you guys need to use Chunking ?

[khimaros]

try increasing n_ctx to something like 2048

[lingster]

llm = GPT4All(model=gpt4, n_ctx=2048, n_threads=8, callback_manager=callback_manager, verbose=True)

[alvations]

from pygpt4all.models.gpt4all import GPT4All

model = GPT4All('ggml-gpt4all-l13b-snoozy.bin', n_ctx=2048)

model.generate("Hello world", n_predict=500, new_text_callback=lambda x: print(x, end=""))

[gnosisd]

i am trying to do the same thing with the friend who created the post i want also to use that but for an sql using the local gpt4all but i just cant get it to work.please someone with experience or time to help me with this ?

`import os import sqlite3 import tkinter as tk import tkinter.ttk as ttk from langchain.agents import create_sql_agent from langchain.agents.agent_toolkits import SQLDatabaseToolkit from langchain.sql_database import SQLDatabase from pygpt4all.models.gpt4all import GPT4All

Create a GPT4All model

model_path = 'C:/Users/Ioannis/AppData/Local/nomic.ai/GPT4All/ggml-gpt4all-l13b-snoozy.bin' model = GPT4All(model_path, n_ctx=2048)

Connect to the database and execute the SQL script

conn = sqlite3.connect('C:/Users/Ioannis/gpt4allz/bin/sqlcon/ChatGPT-Tabular-Data-main/Chinook.db') with open('C:/Users/Ioannis/gpt4allz/bin/sqlcon/ChatGPT-Tabular-Data-main/Chinook_Sqlite.sql', 'r', encoding='cp1252', errors='replace') as f: sql_script = f.read() conn.executescript(sql_script) conn.close()

Create the agent executor

db = SQLDatabase.from_uri("sqlite:///C:/Users/Ioannis/gpt4allz/bin/sqlcon/ChatGPT-Tabular-Data-main/Chinook.db") toolkit = SQLDatabaseToolkit(db=db)

agent_executor = create_sql_agent(llm=model, toolkit=toolkit, verbose=True)

Create the UI window

root = tk.Tk() root.title("Chat with your Tabular Data")

Create the text entry widget

entry = ttk.Entry(root, font=("Arial", 14)) entry.pack(padx=20, pady=20, fill=tk.X)

Create the button callback

def on_click(): # Get the query text from the entry widget query = entry.get()

try:
    # Generate text using the GPT4All model
    generated_text = model.generate(query, n_predict=500)
    
    # Run the query using the agent executor
    result = agent_executor.run(query)
    
    # Display the generated text and the result in the text widget
    text.delete("1.0", tk.END)
    text.insert(tk.END, f"Generated Text: {generated_text}\n\nResult: {result}")
except Exception as e:
    # Catch any exceptions, print them, and display a message
    print(f"Error: {e}")
    text.delete("1.0", tk.END)
    text.insert(tk.END, "An error occurred. Please try again.")

Create the button widget

button = ttk.Button(root, text="Chat", command=on_click) button.pack(padx=20, pady=20)

Create the text widget to display the result

text = tk.Text(root, height=10, width=60, font=("Arial", 14)) text.pack(padx=20, pady=20)

Start the UI event loop

root.mainloop() `

[gnosisd]

anyone knows how i can connect an sql database with chat4all? any pointers?

[gnosisd]

well it looks like that chat4all is not buld to respond in a manner as chat gpt to understand that it was to do query in the database.so i think a better mind than mine is needed.in case someone wants to test it out here is my code `import sqlite3 from langchain.llms import GPT4All

Path to the GPT4All model

model_path = "C:/Users/AppData/Local/nomic.ai/GPT4All/ggml-gpt4all-l13b-snoozy.bin"

Path to the SQLite database

db_path = "C:/Users/gpt4allz/bin/db/Chinook.db"

Instantiate the model

model = GPT4All(model=model_path, n_ctx=2048, n_threads=8)

Connect to the SQLite database

conn = sqlite3.connect(db_path)

Create a cursor

cur = conn.cursor()

Generate a SQL query using the model

query = model("SELECT * FROM Albums WHERE name = 'Sgt. Pepper''s Lonely Hearts Club Band'")

Print the generated SQL query

print("Generated SQL query:", query)

Execute the generated query

cur.execute(query)

Fetch all the rows

rows = cur.fetchall()

Print the rows

for row in rows: print(row)

Close the connection

conn.close() ` if you try something more generic in will not undertand at all like give me a description of table .it answers with things that does not do for this job.so good luck and if you do find a way please let me know....

[dosubot[bot]]

Hi, @Tukks! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you raised is about the prompt being too long in the GPT4All example, which results in an error message. There have been suggestions from other users to adjust the n_ctx parameter and use chunking as potential solutions. Some users have even provided code snippets to address this issue. Additionally, there is a discussion about connecting an SQL database with GPT4All.

Before we proceed, we would like to confirm if this issue is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation. Let us know if you have any further questions or concerns.