try/except not working on WorkerLostError and signal 11

[li-dennis]

Hi,

Is there a way to catch the WorkerLostError exception when it does die from a SIGSEGV? It seems like this might be unavoidable in some cases because of OOM issues for certain CPU-intensive tasks. In which case I'd like to log it. But I can't seem to tap into the worker or celery life cycle to do this.

But I haven't been able to try/catch the exception so far, in either celery 4.4.0 or 5.0.0. I've tried modifying the task, and also task_failure.connect, but no luck.

logs:

[2020-10-14 19:23:22,154: ERROR/MainProcess] Process 'ForkPoolWorker-17' pid:48328 exited with 'signal 11 (SIGSEGV)'
[2020-10-14 19:23:22,170: ERROR/MainProcess] Task handler raised error: WorkerLostError('Worker exited prematurely: signal 11 (SIGSEGV).')
[2020-10-14 19:23:22,403: DEBUG/ForkPoolWorker-18] Using selector: KqueueSelector

the relevant celery code:

@app.task(bind=True, base=CalculationTask)
def async_calculate(self, id, **kwargs):
    serializer = get_serializer(id)
    try:
        return serializer.run()
    except Exception as e:
        print("ERROR")
        print(e)

...

@task_failure.connect
def task_fail_handler(sender, result, **kwargs):
    print("task_fail_handler ERROR")

Excuse the print statements :) but for whatever reason, I can't seem to capture this exception. Neither are getting triggered.