readthedocs.org icon indicating copy to clipboard operation
readthedocs.org copied to clipboard

Published 20 hours ago • verified publisher icon readthedocs

Build: handle Celery timeout events

Open humitos opened this issue 2 years ago • 1 comments

Solve the problem reported by Sentry when the Celery task times out: https://sentry.io/organizations/read-the-docs/issues/3038754375/

Each commit has a detailed explanation of the changes

Note to myself: the latest commit tries to handle soft timeout cancelled properly. It's close but it needs a little more of QA because for some reason SIGUSR1 is triggered/catched as well.

humitos avatar Jun 29 '22 14:06 humitos

Note that we are not hitting this too frequently because we have an pretty high timeout due to this: https://github.com/readthedocs/readthedocs.org/blob/634f85bf00ffcab7bc91926cc9bf9cc394d375fb/readthedocs/projects/tasks/utils.py#L95-L100

humitos avatar Jun 30 '22 20:06 humitos

I'm closing this PR for now. We can come back in the future if we consider. The idea is good, but I wasn't able to properly QA on production to be sure it works as I expect.

humitos avatar Aug 09 '23 08:08 humitos