langchain
langchain copied to clipboard
langchain-ai
How to return the text which is after Finished Chain or Final Answer?
System Info
I ran the below code
output = agent.run("what is Grass Type which pokemnon has highest speed and lowest speed?")
The above code gave the below output
> Entering new AgentExecutor chain...
Thought: I need to find the pokemon with the highest and lowest speed that are of type Grass
Action: python_repl_ast
Action Input: df[df['Type 1'] == 'Grass'][['Name', 'Speed']].sort_values('Speed')
Observation: Name Speed
658 Ferroseed 10
651 Foongus 15
659 Ferrothorn 20
207 Sunflora 30
511 AbomasnowMega Abomasnow 30
.. ... ...
556 Serperior 113
607 Whimsicott 116
274 Sceptile 120
551 ShayminSky Forme 127
275 SceptileMega Sceptile 145
[70 rows x 2 columns]
Thought: I now know the pokemon with the highest and lowest speed that are of type Grass
Final Answer: The Grass Type pokemon with the highest speed is SceptileMega Sceptile with 145 speed, and the Grass Type pokemon with the lowest speed is Ferroseed with 10 speed.
> Finished chain.
But I don't need the complete output. I only need text which is after Final Answer: i.e. The Grass Type pokemon with the highest speed is SceptileMega Sceptile with 145 speed, and the Grass Type pokemon with the lowest speed is Ferroseed with 10 speed.
How to get this output? Any ideas?
Who can help?
No response
Information
- [X] The official example notebooks/scripts
- [ ] My own modified scripts
Related Components
- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async
Reproduction
output = agent.run("what is Grass Type which pokemnon has highest speed and lowest speed?"
Expected behavior
I'm just looking to filter out the output content.
Have you tried SteerCode ? This is a generated response
To get the final answer from the output, you can use the appropriate output parser class based on the agent you are using. Here's an example using the SelfAskOutputParser:
from langchain.agents.self_ask_with_search.output_parser import SelfAskOutputParser
output = agent.run("what is Grass Type which pokemnon has highest speed and lowest speed?")
output_parser = SelfAskOutputParser()
parsed_output = output_parser.parse(output)
if isinstance(parsed_output, AgentFinish):
final_answer = parsed_output.data['output']
print(final_answer)
else:
print("Could not find the final answer.")
Replace SelfAskOutputParser with the appropriate output parser class if you are using a different agent, such as ChatOutputParser, MRKLOutputParser, or ReActOutputParser.
Have you tried SteerCode ? This is a generated response
To get the final answer from the output, you can use the appropriate output parser class based on the agent you are using. Here's an example using the SelfAskOutputParser:
from langchain.agents.self_ask_with_search.output_parser import SelfAskOutputParser output = agent.run("what is Grass Type which pokemnon has highest speed and lowest speed?") output_parser = SelfAskOutputParser() parsed_output = output_parser.parse(output) if isinstance(parsed_output, AgentFinish): final_answer = parsed_output.data['output'] print(final_answer) else: print("Could not find the final answer.")Replace SelfAskOutputParser with the appropriate output parser class if you are using a different agent, such as ChatOutputParser, MRKLOutputParser, or ReActOutputParser.
It is returning an error. Below is the error
Thought: I now know the pokemon with the highest and lowest speed that are of type Grass
Final Answer: The Grass Type pokemon with the highest speed is SceptileMega Sceptile with 145 speed and the Grass Type pokemon with the lowest speed is Ferroseed with 10 speed.
> Finished chain.
---------------------------------------------------------------------------
OutputParserException Traceback (most recent call last)
Cell In[11], line 5
3 output = agent.run("what is Grass Type which pokemnon has highest speed and lowest speed?")
4 output_parser = SelfAskOutputParser()
----> 5 parsed_output = output_parser.parse(output)
7 if isinstance(parsed_output, AgentFinish):
8 final_answer = parsed_output.data['output']
File ~\anaconda3\lib\site-packages\langchain\agents\self_ask_with_search\output_parser.py:15, in SelfAskOutputParser.parse(self, text)
13 finish_string = "So the final answer is: "
14 if finish_string not in last_line:
---> 15 raise OutputParserException(f"Could not parse output: {text}")
16 return AgentFinish({"output": last_line[len(finish_string) :]}, text)
18 after_colon = text.split(":")[-1]
OutputParserException: Could not parse output: The Grass Type pokemon with the highest speed is SceptileMega Sceptile with 145 speed and the Grass Type pokemon with the lowest speed is Ferroseed with 10 speed.
i find it : https://github.com/hwchase17/langchain/commit/7388248b3eeb6966e9139d60d786416c6cb73d66#diff-3c17bbb71875ec39f856668e30f545040c1db1a4b82f5f251edecfd66e5ff979
i find it : 7388248#diff-3c17bbb71875ec39f856668e30f545040c1db1a4b82f5f251edecfd66e5ff979
Hey! The code in the link which you have shared looks complex and it is showing raw notebook elements. Can you explain how to get the only answer i.e. context after finished chain?
i find it : 7388248#diff-3c17bbb71875ec39f856668e30f545040c1db1a4b82f5f251edecfd66e5ff979
Hey! The code in the link which you have shared looks complex and it is showing raw notebook elements. Can you explain how to get the only answer i.e. context after finished chain?
look this: https://python.langchain.com/en/latest/modules/agents/streaming_stdout_final_only.html?highlight=Only%20streaming%20final%20agent%20output
I tried it out and it seems that the latest version doesn't have a Final Answer tag, so it didn't successfully output. If I figure it out, I'll let you know.
------------------------------------update ------------------------------------------------------------------------------ I have solved this problem, here is my code:
from typing import Any, Union
from asyncio.queues import Queue
from langchain.callbacks.base import AsyncCallbackHandler
from typing import Any, Dict, List, Optional
from langchain.schema import LLMResult
import json
from langchain.callbacks.streaming_stdout_final_only import FinalStreamingStdOutCallbackHandler
DEFAULT_ANSWER_PREFIX_TOKENS = ["\n", "AI", ":"]
class CustomAsyncCallBackHandler(AsyncCallbackHandler):
queue: Queue
def __init__(self,queue:Queue, answer_prefix_tokens: Optional[List[str]] = None) -> None:
super().__init__()
if answer_prefix_tokens is None:
answer_prefix_tokens = DEFAULT_ANSWER_PREFIX_TOKENS
self.answer_prefix_tokens = answer_prefix_tokens
self.last_tokens = [""] * len(answer_prefix_tokens)
self.answer_reached = False
self.queue=queue
async def put_message(self,json_str):
await self.queue.put(json.dumps(json_str))
# # 等待所有消息发送完成
await self.queue.join()
async def on_llm_start(
self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
) -> None:
"""Run when LLM starts running."""
self.answer_reached = False
async def on_llm_new_token(self, token: str, **kwargs) -> None:
# Remember the last n tokens, where n = len(answer_prefix_tokens)
self.last_tokens.append(token)
if len(self.last_tokens) > len(self.answer_prefix_tokens):
self.last_tokens.pop(0)
# Check if the last n tokens match the answer_prefix_tokens list ...
if self.last_tokens == self.answer_prefix_tokens:
self.answer_reached = True
# Do not print the last token in answer_prefix_tokens,
# as it's not part of the answer yet
return
# ... if yes, then print tokens from now on
if self.answer_reached:
response = {"stat": "SUCCESS", "message": str(token)}
await self.put_message(response)
async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
"""Run when LLM ends running."""
if self.answer_reached:
response = {"stat": "SUCCESS", "message": "[DONE]"}
#sys.stdout.write(str(response))
#sys.stdout.flush()
print(str(response))
await self.put_message(response)
async def on_llm_error(
self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any
) -> None:
"""Run when LLM errors."""
to use:
llm=ChatOpenAI(temperature=0.5, model_name="gpt-3.5-turbo", verbose=False,streaming=True,callbacks=[CustomAsyncCallBackHandler(message_queue)])
I used WebSocket, you can remove it if you don't need it
model_name="gpt-3.5-turbo"
Below is the code , which i'm working on
import pandas as pd
from IPython.display import Markdown, display
from langchain.agents import create_csv_agent
from langchain.chat_models import ChatOpenAI
import os
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.llms import OpenAI
from langchain.callbacks.base import AsyncCallbackHandler
from typing import Any, Dict, List, Optional
from langchain.schema import LLMResult
import json
from langchain.callbacks.streaming_stdout_final_only import FinalStreamingStdOutCallbackHandler
os.environ["OPENAI_API_KEY"] = ""
def read_file(file_path, file_extension):
if file_extension == 'csv':
df = pd.read_csv(file_path)
elif file_extension == 'xlsx':
df = pd.read_excel(file_path)
elif file_extension == 'json':
df = pd.read_json(file_path)
elif file_extension == 'gbq':
df = pd.read_gbq(file_path)
else:
raise ValueError("Unsupported file format. Only .xlsx, .csv, .json, and .gbq files are supported.")
return df
def run_code(file_path, file_extension):
df = read_file(file_path, file_extension)
# Load the agent
agent = create_csv_agent(OpenAI(temperature=0), model_name="gpt-3.5-turbo", "pokemon.csv", verbose=False,
streaming=True, callbacks=[CustomAsyncCallBackHandler(message_queue)])
gpt4_agent = create_csv_agent(
ChatOpenAI(temperature=0, model_name="gpt-4"), "pokemon.csv", verbose=True
)
while True:
user_input = input("Enter your question (type 'exit' to end): ")
if user_input.lower() == 'exit':
break
agent.run(user_input)
file_path = 'https://gist.githubusercontent.com/armgilles/194bcff35001e7eb53a2a8b441e8b2c6/raw/92200bc0a673d5ce2110aaad4544ed6c4010f687/pokemon.csv'
file_extension = 'csv'
run_code(file_path, file_extension)
The above code is returning the below output
ModuleNotFoundError Traceback (most recent call last)
Cell In[3], line 14
12 from langchain.schema import LLMResult
13 import json
---> 14 from langchain.callbacks.streaming_stdout_final_only import FinalStreamingStdOutCallbackHandler
16 os.environ["OPENAI_API_KEY"] = ""
19 def read_file(file_path, file_extension):
ModuleNotFoundError: No module named 'langchain.callbacks.streaming_stdout_final_only'
I'm unsure why it is returning import error
Perhaps you need to upgrade your
langchain.
and
It seems that you don't need websocket, so it should be removed,like this:
from typing import Any, Union
from asyncio.queues import Queue
from langchain.callbacks.base import AsyncCallbackHandler
from typing import Any, Dict, List, Optional
from langchain.schema import LLMResult
import json
from langchain.callbacks.streaming_stdout_final_only import FinalStreamingStdOutCallbackHandler
DEFAULT_ANSWER_PREFIX_TOKENS = ["\n", "AI", ":"]
class CustomAsyncCallBackHandler(AsyncCallbackHandler):
def __init__(self, answer_prefix_tokens: Optional[List[str]] = None) -> None:
super().__init__()
if answer_prefix_tokens is None:
answer_prefix_tokens = DEFAULT_ANSWER_PREFIX_TOKENS
self.answer_prefix_tokens = answer_prefix_tokens
self.last_tokens = [""] * len(answer_prefix_tokens)
self.answer_reached = False
async def on_llm_start(
self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
) -> None:
"""Run when LLM starts running."""
self.answer_reached = False
async def on_llm_new_token(self, token: str, **kwargs) -> None:
# Remember the last n tokens, where n = len(answer_prefix_tokens)
self.last_tokens.append(token)
if len(self.last_tokens) > len(self.answer_prefix_tokens):
self.last_tokens.pop(0)
# Check if the last n tokens match the answer_prefix_tokens list ...
if self.last_tokens == self.answer_prefix_tokens:
self.answer_reached = True
# Do not print the last token in answer_prefix_tokens,
# as it's not part of the answer yet
return
# ... if yes, then print tokens from now on
if self.answer_reached:
print(token)
async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
"""Run when LLM ends running."""
if self.answer_reached:
print(str(token))
async def on_llm_error(
self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any
) -> None:
"""Run when LLM errors."""
callbacks=[CustomAsyncCallBackHandler()])
I've implemented @reinershir's code into my custom handler (I am using FastAPI so I'm not handling tokens with print(str(token)) or any stdout methods. It works great, but there is one problem -- it's sending the rest of the "final answer" JSON after the actual string response.
For example,
What is the capital of USA?
The capital of USA is Washington D.C."
}
''' <-single quotes instead of backticks so markdown doesn't format it
So, how do we find the stop sequence and stop sending tokens to the client if the stop sequence is encountered? For example, the agent could be finished with the response, but there is still a double quote, a bracket, and three backticks.
I had a couple of ideas for how to go around this, but none of these seem like ideal solutions:
-
Create a token "queue" and delay sending them out for x iterations: I suppose we could have a "queue" of tokens to go out, and if the stop sequence is ever in the queue, then dump it and stop everything. Otherwise, send the tokens in the queue in a delayed fashion ie tokens added to the queue go out after 3 iterations in the queue. This could get really messy and unpredictable though.
- We can't just look for the next double quote in the sequence because if the agent's response included a double quote there would be a premature stop.
- We also can't look for a double-quote then a bracket then three backticks, since the agent's response could have included some JSON code (if the user asked it to do something related to JSON code).
-
Adjusting the prompt to tell the agent to include a pre-determined stop token, like '□', but this gets pretty dicey and weird as well. I tried this and it obeys maybe about 50% of the time, even at
temperature=0. I would not like to rely on the LLM deciding whether to include a stop token.
Is there something I'm missing or am I completely overcomplicating this? It would be nice if an agent could stream a final answer as just a string instead of JSON since it is not actually going into a tool or anything.
Here's my callback handler:
class AsyncStreamCallbackHandler(AsyncCallbackHandler):
def __init__(
self,
send: Sender,
isAgent: bool = False,
answer_prefix_tokens: Optional[List[str]] = [' "', 'Final', ' Answer', '",\n', '"', 'action', '_input', '":', ' "']
):
super().__init__()
self.send = send
self.isAgent = isAgent
self.answer_prefix_tokens = [token.strip() for token in answer_prefix_tokens]
self.last_tokens = [""] * len(answer_prefix_tokens)
def on_llm_start(
self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
) -> None:
"""Run when LLM starts running."""
self.answer_reached = False
self.stop_token_received = False
async def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
"""Rewrite on_llm_new_token to send token to client."""
print("Buffer", self.last_tokens)
if (self.isAgent):
print('LLM Token')
# Remember the last n tokens, where n = len(answer_prefix_tokens)
self.last_tokens.append(token.strip())
if len(self.last_tokens) > len(self.answer_prefix_tokens):
self.last_tokens.pop(0)
# Check if the last n tokens match the answer_prefix_tokens list ...
if self.last_tokens == self.answer_prefix_tokens:
self.answer_reached = True
# Do not print the last token in answer_prefix_tokens,
# as it's not part of the answer yet
return
# ... if yes, then print tokens from now on
if self.answer_reached and not self.stop_token_received:
print("FINAL ANSWER REACHED!")
await self.send(token)
else:
await self.send(token)
async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
"""Run when LLM ends running."""
print("LLM END")
if self.answer_reached:
print(str(response))
jakesteelman
If you want to do something after the final answer output, then on_llm_end method can achieve it. Just return your end tag here, like this: await self.send('[DONE]').
Or did I misunderstand what you meant?
message_queue Hello reinershir could you explain to me how did you use web socket with your custom callback
like here, from where i can get message_queue and how i can use it CustomAsyncCallBackHandler(message_queue)
message_queue Hello reinershir could you explain to me how did you use web socket with your custom callback like here, from where i can get message_queue and how i can use it CustomAsyncCallBackHandler(message_queue)
I have written it on my personal blog, you can refer to it: https://reiner.host/posts/8f0289a1.html
If you want the ouput of your langchain agent try:
input = text + "ChatGPT make summary of these texts together into one in an harmonized and compelling way:"
zero_shot_agent = initialize_agent( agent="zero-shot-react-description", tools=tools, llm=llm, verbose=True, max_iterations=2 )
output = zero_shot_agent.run(input)
print(output,'output')
See "Logging to file" from: https://python.langchain.com/docs/modules/callbacks/filecallbackhandler It works like a charm. You just need to store the file temporarily and until you fetch the final answer.
