fix: add gevent monkey patch to celery.

[erigo]

Checklist:

[!IMPORTANT]
Please review the checklist below before submitting your pull request.

• [x] Please open an issue before creating a PR or link to an existing issue
• [x] I have performed a self-review of my own code
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

Description

Fix #7573

Type of Change

• [x] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] This change requires a documentation update, included: Dify Document
• [ ] Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
• [ ] Dependency upgrade

Testing Instructions

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• [ ] Test A
• [ ] Test B

[erigo]

Have you tried to set DEBUG system env to true? The app.py has been applied in api/app.py when DEBUG=ture.

Yes, I have set DEBUG=true in .env file.

And even if we want to have event monkey patch, we should not have this in celery extension.

Sorry, could you please tell me where the monkey patch should be applied?

[bowenliang123]

@erigo At the line 6 of api/app.py.

monkey.patch_all()

If DEBUG set to true, the line 6 pathes gevent monkey, before the celery app was initialized at Line 156.

[erigo]

@erigo At the line 6 of api.app.py.

monkey.patch_all()

If DEBUG set to true, the line 6 pathes gevent monkey, before the celery app was initialized at Line 156.

Do you mean that if DEBUG is set to true, the monkey patch won't be applied? Could you please clarify? Thanks! 🙏

[bowenliang123]

Thanks for your correction. And the existed code seems only apply the monkey patch when DEBEUG disabled.

What's your main goal in this PR? And how it fix the issue? Do you want to apply event monky patch anyway, in both cases with DEBUG enabled or disabled?

[erigo]

Thanks for your correction. And the existed code seems only apply the monkey patch when DEBEUG disabled.

What's your main goal in this PR? And how it fix the issue? Do you want to apply event monky patch anyway, in both cases with DEBUG enabled or disabled?

Yes, when I started the Dify application in the testing environment with DEBUG mode enabled, I noticed that Celery tasks were getting stuck because they couldn't acquire resources. I tried running Celery from the command line, and even pressing Ctrl+C couldn't stop the process. However, the tasks ran correctly when using the prefork mode. After testing, it seems that the issue was caused by Celery not being monkey-patched.

see #7573

[bowenliang123]

Do you want to apply event monky patch anyway

If this is what you're asking for, these changes I'm suggested.

• leave celery extension ext_celery unchanged, the current committed changes is applying to the whole process anyway. the locality change with global impact should not be isolatedly placed.
• change the code in api/app.py, applying Gevent's monkey patch in all cases.

And for the proper use of Gevent, we should have a discussion with key committers. cc @takatost @laipz8200

[erigo]

Do you want to apply event monky patch anyway

If this is what you're asking for, these changes I'm suggested.

• leave celery extension ext_celery unchanged, the current committed changes is applying to the whole process anyway. the locality change with global impact should not be isolatedly placed.
• change the code in api/app.py, applying Gevent's monkey patch in all cases.

And for the proper use of Gevent, we should have a discussion with key committers. cc @takatost @laipz8200

Got it, I know where the monkey patch should be applied [Thanks]. I'll wait for the discussion to conclude on whether we should apply the monkey patch in all scenarios before submitting a PR.

[bowenliang123]

Have you joined the Wechat contributor group?

[erigo]

Have you joined the Wechat group of contributors?

Not yet, I haven't found the 'organization' 😂.

[bowenliang123]

cc @crazywoola

Have you joined the Wechat group of contributors?

Not yet, I haven't found the 'organization' 😂.

[laipz8200]

Thanks for your correction. And the existed code seems only apply the monkey patch when DEBEUG disabled. What's your main goal in this PR? And how it fix the issue? Do you want to apply event monky patch anyway, in both cases with DEBUG enabled or disabled?

Yes, when I started the Dify application in the testing environment with DEBUG mode enabled, I noticed that Celery tasks were getting stuck because they couldn't acquire resources. I tried running Celery from the command line, and even pressing Ctrl+C couldn't stop the process. However, the tasks ran correctly when using the prefork mode. After testing, it seems that the issue was caused by Celery not being monkey-patched.

see #7573

Hello, could you provide the command you use to start the Celery worker? It runs well in my environment.

BTW you can check the command in api/.vscode/launch.json.example.

[erigo]

Thanks for your correction. And the existed code seems only apply the monkey patch when DEBEUG disabled. What's your main goal in this PR? And how it fix the issue? Do you want to apply event monky patch anyway, in both cases with DEBUG enabled or disabled?

Yes, when I started the Dify application in the testing environment with DEBUG mode enabled, I noticed that Celery tasks were getting stuck because they couldn't acquire resources. I tried running Celery from the command line, and even pressing Ctrl+C couldn't stop the process. However, the tasks ran correctly when using the prefork mode. After testing, it seems that the issue was caused by Celery not being monkey-patched. see #7573

Hello, could you provide the command you use to start the Celery worker? It runs well in my environment.

BTW you can check the command in api/.vscode/launch.json.example.

I use the following command to start Celery worker in docker-api-1 container: celery -A app.celery worker -P gevent -c 1 --loglevel DEBUG -Q dataset,generation,mail,ops_trace,app_deletion

[laipz8200]

Hi @erigo,

I believe this issue is unlikely to be caused by gevent. Since the pool deals with concurrency, and in the debug mode, the concurrency is typically set to 1.

[erigo]

Hi @erigo,

I believe this issue is unlikely to be caused by gevent. Since the pool deals with concurrency, and in the debug mode, the concurrency is typically set to 1.

OK, Thanks for your help. I'm currently working on PR #7756, and I will look into the issue further.