How to Retrieve Intermediate Results Along with ChatOutput via API?

[sungreong]

Bug Description

Hello,

Currently, when I use the API to run a flow, the response only contains the final ChatOutput. However, I would like to retrieve all intermediate results generated by the components in the flow as well.

Could you let me know if there is a way to achieve this? Specifically:

Is there an option in the API to include intermediate results in the response? If not, are there any recommended modifications or configurations needed on the server or in the flow to enable this? Thank you in advance for your guidance!

Reproduction

def main(): parser = argparse.ArgumentParser(description="""Run a flow with a given message and optional tweaks. Run it like: python .py "your message here" --endpoint "your_endpoint" --tweaks '{"key": "value"}'""", formatter_class=RawTextHelpFormatter) parser.add_argument("message", type=str, help="The message to send to the flow") parser.add_argument("--endpoint", type=str, default=ENDPOINT or FLOW_ID, help="The ID or the endpoint name of the flow") parser.add_argument("--tweaks", type=str, help="JSON string representing the tweaks to customize the flow", default=json.dumps(TWEAKS)) parser.add_argument("--api_key", type=str, help="API key for authentication", default=None) parser.add_argument("--output_type", type=str, default="chat", help="The output type") parser.add_argument("--input_type", type=str, default="chat", help="The input type") parser.add_argument("--upload_file", type=str, help="Path to the file to upload", default=None) parser.add_argument("--components", type=str, help="Components to upload the file to", default=None)

args = parser.parse_args()
try:
  tweaks = json.loads(args.tweaks)
except json.JSONDecodeError:
  raise ValueError("Invalid tweaks JSON string")

if args.upload_file:
    if not upload_file:
        raise ImportError("Langflow is not installed. Please install it to use the upload_file function.")
    elif not args.components:
        raise ValueError("You need to provide the components to upload the file to.")
    tweaks = upload_file(file_path=args.upload_file, host=BASE_API_URL, flow_id=args.endpoint, components=[args.components], tweaks=tweaks)

response = run_flow(
    message=args.message,
    endpoint=args.endpoint,
    output_type=args.output_type,
    input_type=args.input_type,
    tweaks=tweaks,
    api_key=args.api_key
)

print(json.dumps(response, indent=2, ensure_ascii=False))

if name == "main": main()

Expected behavior

chatoutput : { results: { } artifacts: { text_output: { repr: "DVC(Data Version Control)는 데이터 관리, 머신러닝(ML) 파이프라인 자동화 및 실험 관리를 위한 오픈 소스 도구입니다. DVC는 데이터 과학 및 머신러닝 팀..." raw: "DVC(Data Version Control)는 데이터 관리, 머신러닝(ML) 파이프라인 자동화 및 실험 관리를 위한 오픈 소스 도구입니다. DVC는 데이터 과학 및 머신러닝 팀..." type: "text" } } outputs: { text_output: { message: "DVC(Data Version Control)는 데이터 관리, 머신러닝(ML) 파이프라인 자동화 및 실험 관리를 위한 오픈 소스 도구입니다. DVC는 데이터 과학 및 머신러닝 팀..." type: "text" } model_output: { message: "" type: "unknown" } } logs: { text_output: [ ] } messages: [ ] timedelta: null duration: null component_display_name: "OpenAI" component_id: "OpenAIModel-OT8wA" used_frozen_result: false } retriever chunk output results: { } artifacts: { search_results: { repr: "[Data(text_key='text', data={'pk': 454235368250671871, 'file_path': '/app/data/.cache/langflow/9747..." raw: [ 0: { text_key: "text" data: { pk: "454235368250671871" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "이러한 방식으로 DVC는 기존 소프트웨어 엔지니어링 도구(Git, IDE, CI/CD, 클라우드 스토리지 등)와 통합되어 ML 프로젝트 관리를 효율적으로 지원합니다. 지금부터 간..." } default_value: "" } 1: { text_key: "text" data: { pk: "454235368250671870" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "실제로 손을 대지 않으면 이것이 매우 혼란스러울 수 있습니다. 따라서 다음 기사에서는 DVC를 사용하여 기계 학습 데이터를 버전 관리하는 방법을 단계별로 보여드리겠습니다. 그렇다..." } default_value: "" } 2: { text_key: "text" data: { pk: "454235368250671886" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "각 파이프라인이 그래프인 것은 의미하지만 단일 dvc.yaml 파일을 의미하지는 않습니다. DVC는 전체 프로젝트 트리를 확인하고 이러한 파일을 찾아 모든 파이프라인을 다시 구축..." } default_value: "" } 3: { text_key: "text" data: { pk: "454235368250671905" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "메트릭, 플롯, 매개변수 DVC는 YAML, JSON, CSV와 같은 표준 구조화된 파일에 저장된 매개변수, 메트릭 및 플롯 데이터를 추적하고 비교할 수 있으며 이러한 데이터를 ..." } default_value: "" } ] type: "array" } } outputs: { base_retriever: { message: "" type: "unknown" } search_results: { message: [ 0: { text_key: "text" data: { pk: "454235368250671871" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "이러한 방식으로 DVC는 기존 소프트웨어 엔지니어링 도구(Git, IDE, CI/CD, 클라우드 스토리지 등)와 통합되어 ML 프로젝트 관리를 효율적으로 지원합니다. 지금부터 간..." } default_value: "" } 1: { text_key: "text" data: { pk: "454235368250671870" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "실제로 손을 대지 않으면 이것이 매우 혼란스러울 수 있습니다. 따라서 다음 기사에서는 DVC를 사용하여 기계 학습 데이터를 버전 관리하는 방법을 단계별로 보여드리겠습니다. 그렇다..." } default_value: "" } 2: { text_key: "text" data: { pk: "454235368250671886" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "각 파이프라인이 그래프인 것은 의미하지만 단일 dvc.yaml 파일을 의미하지는 않습니다. DVC는 전체 프로젝트 트리를 확인하고 이러한 파일을 찾아 모든 파이프라인을 다시 구축..." } default_value: "" } 3: { text_key: "text" data: { pk: "454235368250671905" file_path: "/app/data/.cache/langflow/97479cbe-ea20-46e9-8d9e-f3ae508e9a7e/2024-12-02_02-24-01_샘플.txt" text: "메트릭, 플롯, 매개변수 DVC는 YAML, JSON, CSV와 같은 표준 구조화된 파일에 저장된 매개변수, 메트릭 및 플롯 데이터를 추적하고 비교할 수 있으며 이러한 데이터를 ..." } default_value: "" } ] type: "array" } vector_store: { message: "" type: "unknown" } } logs: { search_results: [ ] } messages: [ ] timedelta: null duration: null component_display_name: "Milvus" component_id: "Milvus-TzOZR" used_frozen_result: false }

Who can help?

No response

Operating System

ubuntu22.04

Langflow Version

1.0.9

Python Version

None

Screenshot

No response

Flow File

No response

[dosubot[bot]]

@sungreong Hello there! I'm here to help you out with any bugs, questions, or contribution guidance you might need. Let me know how I can assist you!

The Langflow API does have a mechanism to retrieve intermediate results, but it is currently not fully active due to the lack of a frontend solution to display these results. The feature is implemented in the get_result_and_steps function, which checks if the langchain_object has the attribute return_intermediate_steps and sets it to True. However, this is deactivated until a frontend solution is available ^[1].

To include intermediate results in the API response, you would need to modify the backend to ensure that these results are included in the response payload. This involves:

1. Activating Intermediate Steps: Modify the code to ensure that return_intermediate_steps is set to True and that the intermediate steps are collected from the output.

2. Modifying the Response Schema: Update the response schema to include intermediate results. This can be done by modifying the RunResponse class or similar response classes to include a field for intermediate results.

3. Updating Serialization Logic: Ensure that the serialization logic in the RunResponse class or other relevant classes includes the intermediate results in the serialized output ^[2].

These changes would require a good understanding of the Langflow codebase and the ability to modify the backend code to handle and return intermediate results as part of the API response.

To continue talking to Dosu, mention @dosu.

[cbornet]

Do you have the code for the run_flow function ? Is it calling the api/v1/run/{flow_id_or_name} endpoint ? If that's the case, this endpoint has a stream query param that can be set to true to stream results. Does it answer your need ?

[dosubot[bot]]

Hi, @sungreong. I'm Dosu, and I'm helping the langflow team manage their backlog. I'm marking this issue as stale.

Issue Summary

• You are experiencing an issue where the API only returns the final output of a flow.
• You are seeking to obtain all intermediate results from the flow.
• The Langflow API has a mechanism for intermediate results, but it's not fully active due to the lack of a frontend solution.
• @cbornet mentioned the stream query parameter in the api/v1/run/{flow_id_or_name} endpoint, which might help.

Next Steps

• Please let us know if this issue is still relevant to the latest version of the langflow repository. If so, you can keep the discussion open by commenting here.
• If there is no further activity, this issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!