workflow object not found maybe data is expired

[freecae]

当我从1.3.1版本升级到2.0.2后，我在运行工作流的时候出现错误：workflow object not found maybe data is expired。 但是这个错误也不是每次都会遇到，多重试几次会有一次成功运行。 后端具体报错信息如下： [2025-08-26 10:28:36,580: WARNING/MainProcess] [2025-08-26 10:28:36.569073] [ERROR process-34032-138594811364928 bisheng.worker.workflow.tasks:98] - trace=ae9f0910b5b44de791e42da497d4f5d8 continue_workflow error
Traceback (most recent call last):

File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/threading.py", line 973, in _bootstrap
self._bootstrap_inner()
| -> <function Thread._bootstrap_inner at 0x7e0f266e1c60>
-> <Thread(ThreadPoolExecutor-3_0, started 138594811364928)>
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
| -> <function Thread.run at 0x7e0f266e1990>
-> <Thread(ThreadPoolExecutor-3_0, started 138594811364928)>
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
| | | | | -> {}
| | | | -> <Thread(ThreadPoolExecutor-3_0, started 138594811364928)>
| | | -> (<weakref at 0x7e0d8825cf90; to 'ThreadPoolExecutor' at 0x7e0d884cebf0>, <_queue.SimpleQueue object at 0x7e0d884f0770>, None,...
| | -> <Thread(ThreadPoolExecutor-3_0, started 138594811364928)>
| -> <function _worker at 0x7e0f25bb57e0>
-> <Thread(ThreadPoolExecutor-3_0, started 138594811364928)>
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
work_item.run()
| -> <function _WorkItem.run at 0x7e0f25bb5900>
-> <concurrent.futures.thread._WorkItem object at 0x7e0d78543c40>
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
| | | | | -> {}
| | | | -> <concurrent.futures.thread._WorkItem object at 0x7e0d78543c40>
| | | -> (<function fast_trace_task at 0x7e0f24c94040>, ('bisheng.worker.workflow.tasks.continue_workflow', '4e8800cf-4f33-407b-8d2b-a...
| | -> <concurrent.futures.thread._WorkItem object at 0x7e0d78543c40>
| -> <function apply_target at 0x7e0f25bb5e10>
-> <concurrent.futures.thread._WorkItem object at 0x7e0d78543c40>
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/site-packages/celery/concurrency/base.py", line 30, in apply_target
ret = target(*args, **kwargs)
| | -> {}
| -> ('bisheng.worker.workflow.tasks.continue_workflow', '4e8800cf-4f33-407b-8d2b-ad4666054416', {'lang': 'py', 'task': 'bisheng.w...
-> <function fast_trace_task at 0x7e0f24c94040>
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/site-packages/celery/app/trace.py", line 651, in fast_trace_task
R, I, T, Rstr = tasks.trace(
| -> 'bisheng.worker.workflow.tasks.continue_workflow'
-> {'celery.group': <@task: celery.group of bisheng at 0x7e0e6cf9dc90>, 'bisheng.worker.knowledge.file_worker.file_copy_celery':...
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/site-packages/celery/app/trace.py", line 453, in trace_task
R = retval = fun(*args, **kwargs)
| | -> {}
| -> ['ae9f0910b5b44de791e42da497d4f5d8', '1ea1c1282aac4d9bbf3280320bf7f792', '', '1']
-> <@task: bisheng.worker.workflow.tasks.continue_workflow of bisheng at 0x7e0e6cf9dc90>
File "/root/anaconda3/envs/BiShengVENV/lib/python3.10/site-packages/celery/app/trace.py", line 736, in protected_call
return self.run(*args, **kwargs)
| | | -> {}
| | -> ('ae9f0910b5b44de791e42da497d4f5d8', '1ea1c1282aac4d9bbf3280320bf7f792', '', '1')
| -> <staticmethod(<function continue_workflow at 0x7e0da1a83d00>)>
-> <@task: bisheng.worker.workflow.tasks.continue_workflow of bisheng at 0x7e0e6cf9dc90>

File "/workspaces/bisheng/src/backend/bisheng/worker/workflow/tasks.py", line 107, in continue_workflow
_continue_workflow(unique_id, workflow_id, chat_id, user_id)
| | | | -> '1'
| | | -> ''
| | -> '1ea1c1282aac4d9bbf3280320bf7f792'
| -> 'ae9f0910b5b44de791e42da497d4f5d8'
-> <function _continue_workflow at 0x7e0da1a83c70>

File "/workspaces/bisheng/src/backend/bisheng/worker/workflow/tasks.py", line 84, in _continue_workflow
raise Exception('workflow object not found maybe data is expired')

Exception: workflow object not found maybe data is expired

[Vincent990413]

是的，您好，我也曾遇到过这个问题很多次。目前不太清楚是否与 celery 的工作流队列有关，或者是模型的质量问题有关，期待有人能发现对应的问题和解决方案。 Yes, hello. I have also encountered this problem many times. Currently, it's not clear whether it's related to celery's workflow queue or the quality of the model. I hope someone can find the corresponding problem and solution.

[zgqgit]

1、确保工作流的执行worker只有一个进程在运行 2、工作流运行期间worker重启会导致这个问题发生

[Vincent990413]

你好，是否是因为我这边的启动命令有误： python.exe -m celery -A bisheng.worker.main worker -l info --pool=solo -Q workflow_celery,knowledge_celery,celery

[zgqgit]

应该是好的，是否在工作流执行期间重启过celery进程？

[Vincent990413]

好的，应该是有的，下次我会注意你说到的两个点。谢谢回复，感谢。

[Vincent990413]

应该是好的，是否在工作流执行期间重启过celery进程？

您好作者，我想知道，如何遇到这个问题，如果是docker运行的，应该如何解决呢？是需要重新docker-compose down然后up吗？之后如何去避免这个问题呢？我是使用Docker运行的。

[yogboot]

我用的 celery -A bisheng.worker.main worker -l info --pool=solo -Q workflow_celery,knowledge_celery,celery，这个命令，没有重启也偶尔会出现这个问题，官方能优化下不

[zgqgit]

目前工作流的运行时都是维护在内存中的，所以如果你的工作流运行到了输入节点时。此时无论什么原因导致的worker服务重启都会出现这个报错。后边版本会考虑将工作流的运行时持久化存储，这样就可以避免这个问题了

[Vincent990413]

感谢开发大大百忙之中的回复，后续是会打算用redis进行持久化吗？我看有一行代码注释了。

---Original--- From: "GuoQing @.> Date: Wed, Sep 3, 2025 10:47 AM To: @.>; Cc: @.@.>; Subject: Re: [dataelement/bisheng] workflow object not found maybe data isexpired (Issue #1576)

zgqgit left a comment (dataelement/bisheng#1576)

目前工作流的运行时都是维护在内存中的，所以如果你的工作流运行到了输入节点时。此时无论什么原因导致的worker服务重启都会出现这个报错。后边版本会考虑将工作流的运行时持久化存储，这样就可以避免这个问题了

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[zgqgit]

是的

[yogboot]

好的，期待^-^