Add support for gevent

[bartsanchez]

Question

Hello,

I have a Flask app (using Flask-SqlAlchemy extension) that I am serving with Gunicorn.

If I try to use the connector (in the way it's defined for Flask-Sqlalchemy section, see attached code) using the worker class "gevent" I get the following error:

google.cloud.sql.connector.exceptions.ConnectorLoopError: Connector event loop is running in current thread! Event loop must be attached to a different thread to prevent blocking code!

However, using "gthread" (threads from the OS, if I am not wrong) and thus not monkey patching the code gives me no problems and the app starts properly (I guess it's because there's no event loop, so it's fine).

I'd like to know whether it's me doing something incorrect or if maybe it's because this connector is incompatible with monkey patching the code and using greenlet.

Thank you!

Code

def getconn():
    conn = connector.connect(
        "FOO",
        "pg8000",
        user="MYUSER",
        enable_iam_auth=True,
        db="MYDB",
    )
    return conn

Additional Details

No response

[jackwotherspoon]

Hi there @bartsanchez! Thanks for raising a question for the Cloud SQL Python Connector 😄

I'm not familiar with "gevent" so I'm not sure I completely follow. I think I need more information from you in order to get a good grasp of your use-case if that is okay with you. If you are monkey patching the library and that is causing the error then I'd need to see the code you are monkey-patching? (if it is on github somewhere can you link it)

The Cloud SQL Python Connector allows control of its event loop through the following Connector(loop=loop) however we don't recommend users to use this unless in rare advanced use-cases (which this may be). By default the Connector will spin up an event loop and assign it to a background thread in order to perform background refreshes that allow for successful connection and IAM database authentication. https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/blob/3e2bfbc2fe721b3d3521cb6d3d73936d0756ce7b/google/cloud/sql/connector/connector.py#L96-L108

For synchronous drivers such as pg8000 if connector.connect is called from the same event loop as the one running in the background thread you will get the error you are seeing as a preventative measure, otherwise the connector.connect call will hang.

Once I know a bit more about your use-case and what kind of monkey patching you are doing (code example) I can provide more insights 😄

[bartsanchez]

Hello Jack! Thanks for the quick response.

Gevent is a library to get asynchrony ("coroutine-based Python networking library") without the need to modify your non-async code by "monkey-patching" I/O bound, "blocking parts with compatible cooperative counterparts" ( taken from this web ).

So, I am not monkey patching directly, it's Gunicorn the one that is monkey patching all the possible functions when it starts the processes.

When your requests are mostly I/O bound tasks (that is my case), this is quite convenient since "it will allow it to scale to potentially thousands of concurrent requests on a single process".

From this part of the documentation, I can see that all greenlets are running in the same OS thread, which makes me think, after your answer, that it could be the case that then they are colliding with the event loop living in the background thread (or maybe there's not even a background thread, given the Gunicorn configuration for using gevent and the supposed new thread is running in a greenlet instead? and then the connector detects the collision and raise the error.? this is the part I get mostly lost).

I was thinking that maybe you or someone could have some experience using the connector for "monkey-patched" gevent code, but otherwise I think I can safely switch to use threads from the OS with Gunicorn (where the connector just works fine).

Sorry for my imprecision explaining the concepts but that's also why I raised the question in the first time, to try to bring some light about what could be happening and understanding if the usage of greenlets and the cloud-sql-python-connector was compatible (thus I was not configuring it properly) or maybe I was doing something conceptually wrong instead.

[jackwotherspoon]

@bartsanchez Thanks for the reply! Seems like our library conflicts with gevent from what you found. We try to optimize our synchronous code which is why we spin up the background thread that runs async tasks.

I'll leave this open as a reminder for me to give gevent a try with the Connector myself to see if there is way to have the two work nicely together.

[bartsanchez]

Nice! Thank you!

Let me know if I can help somehow. I'll be glad.

[magnus0422]

Hello @jackwotherspoon I'm having the same issue and please let me know if you have any updates on it. Thanks.

[enocom]

@magnus0422 are you using Flask?

A little toy reproduction would help us understand what's going wrong here.

[magnus0422]

@enocom Yes, I'm using Flask. Here is the discussion on stackoverflow. https://stackoverflow.com/questions/77907693/how-to-use-eventlet-monkey-patch-and-google-cloud-sql-connector-together/77908089?noredirect=1#comment137349256_77908089

[enocom]

Thanks, @magnus0422. I don't see why we can't just properly fix this. Updated the title to make it clear that this library needs some work to function with gevent.

[magnus0422]

@enocom Hopefully you can provide me the solution soon. Thanks

[jackwotherspoon]

Was able to reproduce this with eventlet with the following basic Flask application:

import eventlet
import sqlalchemy
from flask import Flask
from google.cloud.sql.connector import Connector

eventlet.monkey_patch(socket=True)

# initialize Python Connector object
connector = Connector()


# Python Connector database connection function
def getconn():
    conn = connector.connect(
        "project:region:instance",  # Cloud SQL Instance Connection Name
        "pg8000",
        user="my-user",
        password="my-pass",
        db="postgres",
    )
    return conn


def create_app():
    app = Flask(__name__)

    pool = sqlalchemy.create_engine("postgresql+pg8000://", creator=getconn)

    @app.route("/")
    def hello_time():
        with pool.connect() as conn:
            time = conn.execute(sqlalchemy.text("SELECT NOW()")).fetchone()
        return str(time[0])

    return app

Command to deploy the app:

gunicorn -k eventlet 'app:create_app()'

Will update post when a potential solution is found.

[magnus0422]

Thank you @jackwotherspoon