Monkey-patching ssl after ssl has already been imported

[buxizhizhoum]

Recently we encountered a problem, if we use requests in a flask project, it will finally trigger an warning: /Users/buxizhizhoum/.virtualenvs/gunicorn_monkey_patch/lib/python3.6/site-packages/gunicorn/workers/ggevent.py:38: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (/Users/bytedance/.virtualenvs/gunicorn_monkey_patch/lib/python3.6/site-packages/urllib3/util/__init__.py)', 'urllib3.util.ssl_ (/Users/bytedance/.virtualenvs/gunicorn_monkey_patch/lib/python3.6/site-packages/urllib3/util/ssl_.py)']. This could be reproduced with the code below: the app.py file

#!/usr/bin/python
# -*- coding: utf-8 -*-
from flask import Flask
import requests


app = Flask(__name__)


@app.route("/test")
def test():
    requests.get("https://www.baidu.com")
    return "test"


if __name__ == "__main__":
    app.run(debug=True)

the command to start it: gunicorn -w 8 --worker-class gevent --preload -b 0.0.0.0:5000 app:app

A simplified start procedure of gunicorn is: 1.parse config 2.load app 3.init worker gunicorn imported the ssl module(config.py in gevent) when loading config，however the monkey patch is called when init the worker which is definitely after import of ssl, this is why we get the warning that mentioned above.

We tired 3 ways to solve this problem

1. monkey patch at the beginning of app.py
2. use a config file for gunicorn, and monkey patch it at the beginning of config file.
3. modify GeventWorker.patch in gevent.ggevent.py, change monkey.patch_all() to monkey.patch_all(ssl=False)

the method 1 has no use, since import ssl is before load app the method 2 could work, if we regardless of the drawback that we patched 2 times, method 2 is almost the best work around that I could find. the method 3 could also work, the drawback is ssl is blocking now.

And besides all of above, I just wonder could we provide a choice that allow users to set ssl patch to False, and it could solve the problem, considering the fact that patch ssl after import may cause severe problems, leave ssl not patched might be an solution for someone that could accept the performance lose?

And if wen change app.py file to lines below, it could trigger RecursionError: maximum recursion depth exceeded while calling a Python object app.py that could trigger RecursionError

#!/usr/bin/python
# -*- coding: utf-8 -*-
from flask import Flask
import requests

app = Flask(__name__)

from requests.packages.urllib3.util.ssl_ import create_urllib3_context
ctx = create_urllib3_context()


@app.route("/test")
def test():
    requests.get("https://www.baidu.com")
    return "test"


if __name__ == "__main__":
    app.run(debug=True)

When we start it with gunicorn and call http://127.0.0.1/test it will raise RecursionError, which is mentioned at Gunicorn RecursionError with gevent and requests in python 3.6.2

The environment

Flask             1.1.2
gevent            1.5.0
gunicorn          20.1.0
requests          2.25.1

[panteparak]

create a file called geventlet-config.py with the following content

try:
    import gevent.monkey
    gevent.monkey.patch_all()
except ImportError:
    pass

now add the following argument to your gunicorn. --config geventlet-config.py

So if you started gunicorn using gunicorn --config geventlet-config.py -w 8 --worker-class gevent -b 0.0.0.0:5000 app:app. The patch should be applied

[marklitle]

delete --preload