prefect icon indicating copy to clipboard operation
prefect copied to clipboard

Flow run completed and tasks in "Running" state

Open lucasbelo777 opened this issue 4 months ago • 7 comments

Bug summary

Flows runs are keeping the tasks in running state even with the flow run already completed

Image Image

The logs says the task finished in state completed but seems as running in the UI and in the database

Image Image

Version info

Version:             3.4.13
API version:         0.8.4
Python version:      3.12.11
Git commit:          2b0ea9be
Built:               Thu, Aug 14, 2025 09:13 PM
OS/Arch:             linux/aarch64
Profile:             ephemeral
Server type:         ephemeral
Pydantic version:    2.11.7
Server:
  Database:          postgresql
Integrations:
  prefect-kubernetes: 0.6.4
  prefect-redis:     0.2.4

Additional context

No response

lucasbelo777 avatar Aug 26 '25 19:08 lucasbelo777

Thanks for the issue @lucasbelo777! Do you have an MRE that you can share so that we can reproduce this issue?

desertaxle avatar Aug 26 '25 19:08 desertaxle

yes.

Here is an example with a dummy flow:

Image

Code:


@task()
async def do_work(wait_time: int):
    logger.info("doing some work")
    await asyncio.sleep(wait_time)
    return {"message": "dummy message"}


@prefect_flow(name="dummy_flow")
async def flow(report_id: str, wait_time: int = 5):
    await do_work(wait_time)

lucasbelo777 avatar Aug 26 '25 19:08 lucasbelo777

Also flows finishing in state running.

Image

lucasbelo777 avatar Aug 27 '25 14:08 lucasbelo777

@lucasbelo777 I haven't been able to reproduce this issue yet, but I suspect it might be related to the fact that you're running an ephemeral server. The server might shut down before it receives the completed task run event.

Could you please let me know how often this issue occurs, and do you have any logs from a run where you observed the issue?

desertaxle avatar Aug 27 '25 15:08 desertaxle

mark I have met the same problem

fortunerains avatar Oct 27 '25 10:10 fortunerains

@lucasbelo777 I haven't been able to reproduce this issue yet, but I suspect it might be related to the fact that you're running an ephemeral server. The server might shut down before it receives the completed task run event.@lucasbelo777 我目前还未能复现这个问题,但我怀疑这可能与你正在运行一个临时服务器有关。服务器可能在接收到完成的任务运行事件之前就关闭了。

Could you please let me know how often this issue occurs, and do you have any logs from a run where you observed the issue?能否告知这个问题发生的频率,以及你是否有一个观察到问题的运行日志?

It will appear when I call the submit 1000 subtask now and I use local prefect server deployed docker(postgresql + service+ server)

fortunerains avatar Oct 27 '25 10:10 fortunerains

Image Image

fortunerains avatar Oct 27 '25 11:10 fortunerains