langgenius
Advanced-chat API returns empty metadata
Self Checks
- [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
- [X] I have searched for existing issues search for existing issues, including closed ones.
- [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
- [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
- [X] Please do not modify this template :) and fill in all the required fields.
Dify version
0.8.0
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
- Create an advanced-chat type application.
- Use the API to call the application.
- Set the "response_mode" to "blocking" in the application settings.
- Observe the returned metadata.
✔️ Expected Behavior
The returned metadata should contain information about the chat response, similar to previous versions(0.7.3).
❌ Actual Behavior
The returned metadata is an empty dictionary.
The issue you're experiencing with the Advanced-chat API returning empty metadata when the "response_mode" is set to "blocking" in Dify version 0.8.0 is due to the request potentially being interrupted if the process takes too long. Specifically, Cloudflare restrictions will interrupt the request without a return after 100 seconds.
To avoid this issue, it is recommended to use the streaming mode, which returns results incrementally and is less likely to be interrupted [1][2][3].
If you need to use the blocking mode, you can adjust the GUNICORN_TIMEOUT parameter to prevent the interruption. This parameter controls the request handling timeout and defaults to 200 seconds. It is recommended to set this to 360 seconds to support longer SSE (Server-Sent Events) connection times [4][5].
To continue talking to Dosu, mention @dosu.
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
Did you turn on the switch of reference function?
yes, I tried both cases.
0.8.0
'event': 'message', 'task_id': '2019d74d-91c0-4522-a8bd-47027a816bdd', 'id': 'a64575b6-f95d-4eeb-9c34-9638cb39ab83', 'message_id': 'a64575b6-f95d-4eeb-9c34-9638cb39ab83', 'conversation_id': '514ced6b-d578-4411-a4a4-1378516e68bb', 'mode': 'advanced-chat', 'answer': 'XXXX', 'metadata': {}, 'created_at': 1726043873}
0.7.3
'event': 'message', 'task_id': 'f817d15f-fa12-4ff8-baf2-dd55d1107db5', 'id': '76fe249b-4be4-4de8-81ce-641370b76655', 'message_id': '76fe249b-4be4-4de8-81ce-641370b76655', 'conversation_id': 'c2b27526-3822-4c95-b2e4-b71c90a44de7', 'mode': 'advanced-chat', 'answer': 'XXXX', 'metadata': {'usage': {'prompt_tokens': 24, 'prompt_unit_price': '1', 'prompt_price_unit': '0.000001', 'prompt_price': '0.0000240', 'completion_tokens': 876, 'completion_unit_price': '2', 'completion_price_unit': '0.000001', 'completion_price': '0.0017520', 'total_tokens': 900, 'total_price': '0.0017760', 'currency': 'RMB', 'latency': 32.933926465000695}}, 'created_at': 1726046447}
I tried in the cloud.
curl -X POST 'https://api.dify.ai/v1/chat-messages' \
--header 'Authorization: Bearer KEY HERE' \
--header 'Content-Type: application/json' \
--data-raw '{
"inputs": {},
"query": "What are the specs of the iPhone 13 Pro Max?",
"response_mode": "blocking",
"conversation_id": "",
"user": "abc-123"
}'
{"event": "message", "task_id": "be132aa7-0cb1-4718-be43-62cee4c8907a", "id": "d1476838-844a-4b26-8732-bc4cb2391ff2", "message_id": "d1476838-844a-4b26-8732-bc4cb2391ff2", "conversation_id": "ad60892a-aff3-4a3f-83b4-c4b343de5e93", "mode": "chat", "answer": "The iPhone 13 Pro Max, released in September 2021, comes with several notable specifications:\n\n### Design and Build\n- **Dimensions**: 160.8 x 78.1 x 7.65 mm\n- **Weight**: 238 grams\n- **Materials**: Surgical-grade stainless steel frame, textured matte glass back\n- **Colors**: Graphite, Gold, Silver, Sierra Blue\n\n### Display\n- **Size**: 6.7 inches\n- **Type**: Super Retina XDR OLED\n- **Resolution**: 2778 x 1284 pixels (458 ppi)\n- **Brightness**: Up to 1200 nits (HDR), 1000 nits (typical)\n- **ProMotion**: 120Hz adaptive refresh rate\n\n### Performance\n- **Chip**: A15 Bionic chip\n - **CPU**: 6-core CPU\n - **GPU**: 5-core GPU\n - **Neural Engine**: 16-core\n\n### Camera System\n- **Rear Cameras**:\n - 12 MP Ultra Wide (f/1.8) with 120\u00b0 field of view\n - 12 MP Wide (f/1.5) with sensor-shift optical image stabilization\n - 12 MP Telephoto (f/2.8) with 3x optical zoom\n- **Night mode**, **Deep Fusion**, **Smart HDR 4**, and **Photographic Styles** support\n- **ProRAW and ProRes video recording** capabilities\n- **Front Camera**: 12 MP (f/2.2) with Night mode and 4K video recording\n\n### Battery Life\n- **Video Playback**: Up to 28 hours\n- **Audio Playback**: Up to 95 hours\n- **Fast Charging**: Supports 20W or higher adapter (sold separately) for up to 50% charge in around 30 minutes.\n\n### Storage Options\n- **Variants**: 128GB, 256GB, 512GB, 1TB\n\n### Operating System\n- Initially shipped with iOS 15, compatible with updates to later iOS versions.\n\n### Connectivity\n- **5G capable**\n- **Wi-Fi 6 (802.11ax)**\n- **Bluetooth 5.0**\n- **Ultra Wideband (UWB)** support\n\n### Other Features\n- **Face ID** for secure authentication\n- **Ceramic Shield front cover** for increased durability\n- **Dust and water resistance** (IP68)\n\nThis summary covers the key specifications and features you can expect from the iPhone 13 Pro Max. If you\u2019re looking for more detailed specifications or other features, feel free to ask!", "metadata": {"usage": {"prompt_tokens": 20, "prompt_unit_price": "0.15", "prompt_price_unit": "0.000001", "prompt_price": "0.0000030", "completion_tokens": 569, "completion_unit_price": "0.60", "completion_price_unit": "0.000001", "completion_price": "0.0003414", "total_tokens": 589, "total_price": "0.0003444", "currency": "USD", "latency": 1.1214915139134973}}, "created_at": 1726048356}%
It seems ok to me.
I tried in the cloud.
curl -X POST 'https://api.dify.ai/v1/chat-messages' \ --header 'Authorization: Bearer KEY HERE' \ --header 'Content-Type: application/json' \ --data-raw '{ "inputs": {}, "query": "What are the specs of the iPhone 13 Pro Max?", "response_mode": "blocking", "conversation_id": "", "user": "abc-123" }'{"event": "message", "task_id": "be132aa7-0cb1-4718-be43-62cee4c8907a", "id": "d1476838-844a-4b26-8732-bc4cb2391ff2", "message_id": "d1476838-844a-4b26-8732-bc4cb2391ff2", "conversation_id": "ad60892a-aff3-4a3f-83b4-c4b343de5e93", "mode": "chat", "answer": "The iPhone 13 Pro Max, released in September 2021, comes with several notable specifications:\n\n### Design and Build\n- **Dimensions**: 160.8 x 78.1 x 7.65 mm\n- **Weight**: 238 grams\n- **Materials**: Surgical-grade stainless steel frame, textured matte glass back\n- **Colors**: Graphite, Gold, Silver, Sierra Blue\n\n### Display\n- **Size**: 6.7 inches\n- **Type**: Super Retina XDR OLED\n- **Resolution**: 2778 x 1284 pixels (458 ppi)\n- **Brightness**: Up to 1200 nits (HDR), 1000 nits (typical)\n- **ProMotion**: 120Hz adaptive refresh rate\n\n### Performance\n- **Chip**: A15 Bionic chip\n - **CPU**: 6-core CPU\n - **GPU**: 5-core GPU\n - **Neural Engine**: 16-core\n\n### Camera System\n- **Rear Cameras**:\n - 12 MP Ultra Wide (f/1.8) with 120\u00b0 field of view\n - 12 MP Wide (f/1.5) with sensor-shift optical image stabilization\n - 12 MP Telephoto (f/2.8) with 3x optical zoom\n- **Night mode**, **Deep Fusion**, **Smart HDR 4**, and **Photographic Styles** support\n- **ProRAW and ProRes video recording** capabilities\n- **Front Camera**: 12 MP (f/2.2) with Night mode and 4K video recording\n\n### Battery Life\n- **Video Playback**: Up to 28 hours\n- **Audio Playback**: Up to 95 hours\n- **Fast Charging**: Supports 20W or higher adapter (sold separately) for up to 50% charge in around 30 minutes.\n\n### Storage Options\n- **Variants**: 128GB, 256GB, 512GB, 1TB\n\n### Operating System\n- Initially shipped with iOS 15, compatible with updates to later iOS versions.\n\n### Connectivity\n- **5G capable**\n- **Wi-Fi 6 (802.11ax)**\n- **Bluetooth 5.0**\n- **Ultra Wideband (UWB)** support\n\n### Other Features\n- **Face ID** for secure authentication\n- **Ceramic Shield front cover** for increased durability\n- **Dust and water resistance** (IP68)\n\nThis summary covers the key specifications and features you can expect from the iPhone 13 Pro Max. If you\u2019re looking for more detailed specifications or other features, feel free to ask!", "metadata": {"usage": {"prompt_tokens": 20, "prompt_unit_price": "0.15", "prompt_price_unit": "0.000001", "prompt_price": "0.0000030", "completion_tokens": 569, "completion_unit_price": "0.60", "completion_price_unit": "0.000001", "completion_price": "0.0003414", "total_tokens": 589, "total_price": "0.0003444", "currency": "USD", "latency": 1.1214915139134973}}, "created_at": 1726048356}%It seems ok to me.
I tried replicating the environment by pulling a fresh image and deploying version 0.8.0 again. Unfortunately, I encountered the same issue with the metadata being empty when using the advanced-chat mode. It appears that this problem is specific to the advanced-chat mode, as the chat mode does not exhibit this behavior.
@Halflifefa I encountered exactly the problem I am not at all a python developer but after several analyses via chatgpt I think I have identified the problem, this is due to the fact that in advanced-chat mode several LLM can intervene and therefore several metadata of use, and it is probably the same thing for "retriever_resources". here is a solution that I use myself I did basic tests and it works for me, if it can help someone here is the code and the file to modify:
path file to modify : dify/api/core/app/apps/advanced_chat/generate_task_pipeline.py
from decimal import Decimal
def _save_message(self, graph_runtime_state: Optional[GraphRuntimeState] = None) -> None:
"""
Save message.
:return:
"""
self._refetch_message()
self._message.answer = self._task_state.answer
self._message.provider_response_latency = time.perf_counter() - self._start_at
self._task_state.metadata = self._task_state.metadata or {}
if graph_runtime_state:
usage_list = []
total_prompt_tokens = 0
total_completion_tokens = 0
total_price = Decimal('0.0') # Ensure total_price is a Decimal
currency = None
for node_state in graph_runtime_state.node_run_state.node_state_mapping.values():
if node_state.node_run_result and node_state.node_run_result.llm_usage:
llm_usage = node_state.node_run_result.llm_usage
total_prompt_tokens += llm_usage.prompt_tokens
total_completion_tokens += llm_usage.completion_tokens
total_price += llm_usage.total_price # Both are Decimal
currency = llm_usage.currency
usage_list.append({
'node_id': node_state.node_id,
'node_name': node_state.node_id, # Replace with node_state.node_name if available
'prompt_tokens': llm_usage.prompt_tokens,
'completion_tokens': llm_usage.completion_tokens,
'total_tokens': llm_usage.total_tokens,
'total_price': float(llm_usage.total_price), # Convert to float if needed
'currency': llm_usage.currency,
})
if usage_list:
self._task_state.metadata['usage'] = usage_list
self._message.message_tokens = total_prompt_tokens
self._message.answer_tokens = total_completion_tokens
self._message.total_price = float(total_price) # Convert to float if needed
self._message.currency = currency
retriever_resources_list = []
for node_state in graph_runtime_state.node_run_state.node_state_mapping.values():
if node_state.node_run_result and hasattr(node_state.node_run_result, 'retriever_resources'):
retriever_resources = node_state.node_run_result.retriever_resources
if retriever_resources:
retriever_resources_list.extend(retriever_resources)
if retriever_resources_list:
self._task_state.metadata['retriever_resources'] = retriever_resources_list
self._message.message_metadata = json.dumps(jsonable_encoder(self._task_state.metadata))
db.session.commit()
message_was_created.send(
self._message,
application_generate_entity=self._application_generate_entity,
conversation=self._conversation,
is_first_message=self._application_generate_entity.conversation_id is None,
extras=self._application_generate_entity.extras,
)
Example metadata response :
{
"event": "message",
"task_id": "123e4567-e89b-12d3-a456-426614174000",
"id": "0d7c10a0-27b0-4037-affd-08299950097c",
"message_id": "0d7c10a0-27b0-4037-affd-08299950097c",
"conversation_id": "a398624b-c452-407d-a3d5-7f83e600d6ea",
"mode": "advanced-chat",
"answer": "Bonjour ! Comment puis-je vous aider aujourd'hui ?",
"metadata": {
"usage": [
{
"node_id": "node_id",
"node_name": "node_id",
"prompt_tokens": 500,
"completion_tokens": 100,
"total_tokens": 600,
"total_price": 0.0012,
"currency": "USD"
},
{
"node_id": "node_id_2",
"node_name": "node_id_2",
"prompt_tokens": 300,
"completion_tokens": 80,
"total_tokens": 380,
"total_price": 0.0008,
"currency": "USD"
}
],
"retriever_resources": [
{
"position": 1,
"dataset_id": "dataset1",
"dataset_name": "Base de Connaissances",
"document_id": "doc1",
"document_name": "Document 1",
"segment_id": "seg1",
"score": 0.98,
"content": "Contenu du document..."
},
{
"position": 2,
"dataset_id": "dataset2",
"dataset_name": "FAQ",
"document_id": "doc2",
"document_name": "Document 2",
"segment_id": "seg2",
"score": 0.95,
"content": "Contenu du document..."
}
]
},
"created_at": 1728365895
}
@Halflifefa @crazywoola Hi! I'm new to dify. I was wondering what should be the api_key for the Bearer authorization in retrieving conversation history? Is it created at the api key session that looks something like "app-xxxxxxxxxxxxxxxxx7N5y"?
curl -X GET 'http://xxx.xxx.x.xx/v1/messages?user=abc-123&conversation_id='
--header 'Authorization: Bearer {app-xxxxxxxxxxxxxxxxx7N5y}'
But when I tried it, the output is: {"code": "unauthorized", "message": "Access token is invalid", "status": 401}.
Thanks!
