overhangio
fix: run Celery tasks asynchronously in dev mode
In tutor dev, Celery tasks were running eagerly (in-process) rather than being delegated to the worker containers for asynchronous processing. This difference in behavior between tutor dev and tutor local meant that certain code paths and race-condition bugs would not surface when testing in dev.
The cause is that CELERY_ALWAYS_EAGER is set to True in the upstream devstack settings file. That's because Devstack does not support async LMS/CMS workers; it runs all those tasks in-process.
The fix is to simply override CELERY_ALWAYS_EAGER to False. We do this in the common settings file for simplicity, but the change should not affect production mode, which has always had CELERY_ALWAYS_EAGER equal False.
For unit tests, we override CELERY_ALAYS_EAGER back to True, because many of them are written to assume in-process tasks.
I needed this fix in order to test the upcoming Content Libraries V2 changes, which moves all of the update-from-library operations into Celery tasks.
FYI @ormsbee.
It will make troubleshooting celery tasks more difficult for developers (no pdb for them).
@regisb Let me check this out. I think devs should be able to set breakpoints in Celery tasks and then interact with them using tutor dev start [lms|cms]-worker, but I haven't confirmed that yet. As I mentioned in Slack, I've been having trouble with breakpoints in general recently.