Flask-RQ2 icon indicating copy to clipboard operation
Flask-RQ2 copied to clipboard

Disable reuse of `current_app`

Open alexdong opened this issue 4 years ago • 0 comments

The current approach crashes the worker processes because the sqlalchemy connection isn't initiated properly.

The sympton is that the first job was processed beautifully but subsequent jobs failed with the following stack trace:

Exception during reset or similar Traceback (most recent call last): File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 739, in _r ollback_impl self.engine.dialect.do_rollback(self.connection) File "app/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 22 41, in do_rollback dbapi_connection.rollback() MySQLdb._exceptions.OperationalError: (2006, 'MySQL server has gone away') The above exception was the direct cause of the following exception: Traceback (most recent call last): File "app/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 680, in _fin alize_fairy fairy._reset(pool) File "app/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 865, in _res et self._reset_agent.rollback() File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1705, in r ollback self._do_rollback() File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1743, in _ do_rollback self.connection._rollback_impl() File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 741, in _r ollback_impl self._handle_dbapi_exception(e, None, None, None, None) File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _ handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "app/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 399, in ra ise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "app/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in re raise raise value.with_traceback(tb) File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 739, in _r ollback_impl self.engine.dialect.do_rollback(self.connection) File "app/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 22 41, in do_rollback dbapi_connection.rollback() sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2006, 'MySQL server has gone a way') (Background on this error at: http://sqlalche.me/e/e3q8)

The FlaskJob.load_app has the good intention to reuse the current_app but IMHO it's making too many assumptions of how the Flask app is structured.

Since RQ is mainly used in backend processes where performance is not the highest priority, I'd like to suggest we call create_app every time a job is performed. This provides the maximum guarantee on the correctness of the appcontext.

alexdong avatar Sep 25 '19 22:09 alexdong