[QUESTION] Repeated task: Multiple executions when using multiple workers

[LindezaGrey]

Hey there,

when i use repeated task in production with a docker gunicorn/uvicorn image there are multiple instances of the application running, each one with the repeated task. So for example i want to send notifications periodically, the notification will get send multiple times (number of workers) My workaround for this would be to run a separate container with only one worker, which is then responsible for the repeated task. Is my usecase out of scope or is there a better solution?

best regards,

[Cacsjep]

same question here

[razvanavram]

+1

[cca32]

+1

[Vladimir-v1]

Maybe this solution can be useful for someone, we can get first process and execute some task only in this process:

------ Background Tasks --------

@app.on_event("startup") @repeat_every(seconds=20) async def some_task() -> None: parent_process = psutil.Process(os.getppid()) children = parent_process.children(recursive=True) # List of all child processes if children[0].pid == os.getpid(): # Make some job ... app.log.info('Child pid is {}'.format(children[0].pid)) # Show which process will execute job

[nejos97]

This solution is still valid if we ran fastapi on Kubernetes (with 5 pods for example)?

[priyanshu-panwar]

This project seems to be dead now. You can collaborate here : https://github.com/priyanshu-panwar/fastapi-utilities | This is based on the same project. https://pypi.org/project/fastapi-utilities/0.1.0/

[VladimirMbf]

Hi @priyanshu-panwar! Thank you for your code sharing, but I didn't see the part that trying to prevent multiple background task execution. The main goal as I understand is to prevent execution of same tasks from multiple processes that uvicorn can start (clones of your main code + background task, each process will have same task and will execute this task each one = unwanted multiple code execution )

[priyanshu-panwar]

Hi @priyanshu-panwar! Thank you for your code sharing, but I didn't see the part that trying to prevent multiple background task execution. The main goal as I understand is to prevent execution of same tasks from multiple processes that uvicorn can start (clones of your main code + background task, each process will have same task and will execute this task each one = unwanted code executions )

Hi @VladimirMbf Got your point now. Just thinking if this thing can be implemented on the package-level, so that the user doesn't have to worry about it. This thing can be very useful in production.