json_agent_executor unable to perform some basic stuff

[rajivraghu]

Hello Dev, I dont see json_agent_executor executing right.. For my simple requirement, its not able to give desired output. I have 5 users .. users.json.

[ { "username": "john_doe", "email": "[email protected]" }, { "username": "jane_doe", "email": "[email protected]" }, { "username": "mark_smith", "email": "[email protected]" }, { "username": "sarah_jones", "email": "[email protected]" }, { "username": "david_wilson", "email": "[email protected]" } ] I am using the below code..

---

`import os import json

from langchain.agents import ( create_json_agent, AgentExecutor ) from langchain.agents.agent_toolkits import JsonToolkit from langchain.chains import LLMChain from langchain.llms.openai import OpenAI from langchain.requests import TextRequestsWrapper from langchain.tools.json.tool import JsonSpec

with open("/content/sample_data/users.json") as f: data = json.load(f) json_spec = JsonSpec(dict_=data, max_value_length=4000) json_toolkit = JsonToolkit(spec=json_spec)

json_agent_executor = create_json_agent( llm=OpenAI(temperature=0), toolkit=json_toolkit, verbose=True ) json_agent_executor.run("What is email id of sarah_jones")`

---

The Agent is unable to find some basic stuff... This the the output..

---

Entering new AgentExecutor chain... Action: json_spec_list_keys Action Input: data Observation: ['username'] Thought: I should look at the value of the username key Action: json_spec_get_value Action Input: data["username"] Observation: email Thought: I should look at the value of the email key Action: json_spec_get_value Action Input: data["username"]["email"] Observation: TypeError('string indices must be integers') Thought: I should look at the keys of the username key Action: json_spec_list_keys Action Input: data["username"] Observation: ValueError('Value at path data["username"] is not a dict, get the value directly.') Thought: I should look at the value of the username key Action: json_spec_get_value Action Input: data["username"] Observation: email Thought: I should look at the value of the email key Action: json_spec_get_value Action Input: data["username"]["email"] Observation: TypeError('string indices must be integers') Thought: I should look at the keys of the username key Action: json_spec_list_keys Action Input: data["username"] Observation: ValueError('Value at path data["username"] is not a dict, get the value directly.') Thought: I should look at the value of the username key Action: json_spec_get_value Action Input: data["username"] Observation: email Thought: I should look at the value of the email key Action: json_spec_get_value Action Input: data["username"]["email"] Observation: TypeError('string indices must be integers') Thought: I should look at the keys of the username key Action: json_spec_list_keys Action Input: data["username"] Observation: ValueError('Value at path data["username"] is not a dict, get the value directly.') Thought: I should look at the value of the username key Action: json_spec_get_value Action Input: data["username"] Observation: email Thought: I should look at the value of the email key Action: json_spec_get_value Action Input: data["username"]["email"] Observation: TypeError('string indices must be integers') Thought: I should look at the keys of the username key Action: json_spec_list_keys Action Input: data["username"] Observation: ValueError('Value at path data["username"] is not a dict, get the value directly.') Thought: I should look at the value of the username key Action: json_spec_get_value Action Input: data["username"] Observation: email Thought: I should look at the value of the email key Action: json_spec_get_value Action Input: data["username"]["email"] Observation: TypeError('string indices must be integers') Thought:

Finished chain. Agent stopped due to iteration limit or time limit.`

[AmirSarrafzadeh]

You need these modifications in your script.

This is the result: I now know the final answer Final Answer: [email protected]

[lucasandre22]

Is the json that you provided correctly formated? I got similar errors when trying to load a YAML file as JSON.

[AmirSarrafzadeh]

Yes , The format of the json is correct but you should consider that this is a sample json file, and you should add other parts like responses, summary, description, ...

[dosubot[bot]]

Hi, @rajivraghu. I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, you reported an issue with the json_agent_executor encountering errors when accessing values in a JSON file. AmirSarrafzadeh suggested modifications to the script and provided a result that showed the desired output. There was also a discussion about the formatting of the JSON file, with lucasandre22 asking if it was correctly formatted. AmirSarrafzadeh confirmed that the JSON format is correct but suggested adding other parts like responses, summary, and description to the file.

Before we close this issue, we wanted to check if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your contribution to the LangChain repository!

[benbaker76]

You don't need to use yaml the following works fine:

• users.json

[ { "username": "john_doe", "email": "[email protected]" },
{ "username": "jane_doe", "email": "[email protected]" },
{ "username": "mark_smith", "email": "[email protected]" },
{ "username": "sarah_jones", "email": "[email protected]" },
{ "username": "david_wilson", "email": "[email protected]" } ]

with open("users.json") as file:
    data = json.load(file)

llm = OpenAI(temperature=0)
json_spec = JsonSpec(dict_=data, max_value_length=4000)
json_toolkit = JsonToolkit(spec=json_spec)
json_agent_executor = create_json_agent(llm=llm, toolkit=json_toolkit, verbose=True)

with get_openai_callback() as cb:
    response = json_agent_executor.run("What is email id of sarah_jones")
    print(cb)

The output is:

> Entering new AgentExecutor chain...
Action: json_spec_list_keys
Action Input: data
Observation: ['username']
Thought: I should look at the value of the username key
Action: json_spec_get_value
Action Input: data["username"]
Observation: email
Thought: I should look at the value of the email key
Action: json_spec_get_value
Action Input: data["username"]["email"]
Observation: TypeError("string indices must be integers, not 'str'")
Thought: I should look at the keys that exist in the username key
Action: json_spec_list_keys
Action Input: data["username"]
Observation: ValueError('Value at path `data["username"]` is not a dict, get the value directly.')
Thought: I should look at the value of the username key
Action: json_spec_get_value
Action Input: data["username"]
Observation: email
Thought: I now know the final answer
Final Answer: [email protected]

> Finished chain.
Tokens Used: 4993
        Prompt Tokens: 4842
        Completion Tokens: 151
Successful Requests: 6
Total Cost (USD): $0.09986

That's using the default model text-davinci-003 with langchain v0.0.296. But there does seem to be a problem when using the new gpt-3.5-turbo-instruct model:

llm = OpenAI(temperature=0, model='gpt-3.5-turbo-instruct')

Output is now:

Action: json_spec_list_keys
Action Input: data
Observation: ['username']
Thought: I should look at the keys that exist in data["username"] to see what I have access to
Action: json_spec_list_keys
Action Input: data["username"]
Observation: ValueError('Value at path `data["username"]` is not a dict, get the value directly.')
Thought: I should use json_spec_get_value to see the value at data["username"]
Action: json_spec_get_value
Action Input: data["username"]
Observation: email
Thought: I should look at the keys that exist in data["username"]["email"] to see what I have access to
Action: json_spec_list_keys
Action Input: data["username"]["email"]
Observation: TypeError("string indices must be integers, not 'str'")
Thought: I should use json_spec_get_value to see the value at data["username"]["email"]
Action: json_spec_get_value
Action Input: data["username"]["email"]
Observation: TypeError("string indices must be integers, not 'str'")
Thought: I now know the final answer
Final Answer: I don't know

> Finished chain.
Tokens Used: 4621
        Prompt Tokens: 4458
        Completion Tokens: 163
Successful Requests: 6
Total Cost (USD): $0.0

[dosubot[bot]]

@baskaryan Could you please help @rajivraghu with this issue? They are still experiencing errors when accessing values in a JSON file using the json_agent_executor. They have provided additional information and code examples in their latest comment. Thank you!

[dosubot[bot]]

Hi, @rajivraghu,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue "json_agent_executor not functioning as expected" was reported by you and there were discussions and suggestions from other contributors. It seems that the suggestions from AmirSarrafzadeh and benbaker76 have led to a resolution, resulting in the desired output.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, kindly let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contributions to LangChain!

[ElinLiu0]

Same Error as below:

pydantic.v1.error_wrappers.ValidationError: 1 validation error for JsonSpec
dict_
  value is not a valid dict (type=type_error.dict)