Caleb Collins-Parks comments

Results 79 comments of


                                            Caleb Collins-Parks

unpickleable exceptions cause queue to crash

**TL;DR:** Unpickleable errors can cause the celery queue to crash. The queue should log a warning and continue operating.

unpickleable exceptions cause queue to crash

Probably not. I'll create a ticket for fixing it in our jira instance but I'm not sure when I'll have time to get around to it. I would need to...

unpickleable exceptions cause queue to crash

I've been looking into this a bit more but haven't been able to replicate just yet. Quick question: why is celery using pickle? I looked in the docs and found...

unpickleable exceptions cause queue to crash

Here is a simple repo I made for reproducing the issue: https://github.com/caleb15/celeryPickleBugReproduction I included a readme with quick replication steps and the error stacktrace :)

unpickleable exceptions cause queue to crash

versions 3.8.5 and 3.9.0 of what? python?

unpickleable exceptions cause queue to crash

Are you able to reproduce it with python 3.7.7?

unpickleable exceptions cause queue to crash

Yep, for reproducing please use https://github.com/caleb15/celeryPickleBugReproduction

unpickleable exceptions cause queue to crash

> How is this any different in Celery? Not sure, but celery fails so something must be going on :man_shrugging:

unpickleable exceptions cause queue to crash

Thanks for the tip, but we already avoided the issue by logging the exception instead of raising it. Even if Celery finds invalid XML in the task exception, it should...

unpickleable exceptions cause queue to crash

Fair enough. I'm not the one who coded that so I don't take any offense :D