Flask-RQ2
Flask-RQ2 copied to clipboard
Disable reuse of `current_app`
The current approach crashes the worker processes because the sqlalchemy connection isn't initiated properly.
The sympton is that the first job was processed beautifully but subsequent jobs failed with the following stack trace:
Exception during reset or similar Traceback (most recent call last): File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 739, in _r ollback_impl self.engine.dialect.do_rollback(self.connection) File "app/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 22 41, in do_rollback dbapi_connection.rollback() MySQLdb._exceptions.OperationalError: (2006, 'MySQL server has gone away') The above exception was the direct cause of the following exception: Traceback (most recent call last): File "app/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 680, in _fin alize_fairy fairy._reset(pool) File "app/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 865, in _res et self._reset_agent.rollback() File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1705, in r ollback self._do_rollback() File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1743, in _ do_rollback self.connection._rollback_impl() File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 741, in _r ollback_impl self._handle_dbapi_exception(e, None, None, None, None) File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _ handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "app/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 399, in ra ise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "app/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 153, in re raise raise value.with_traceback(tb) File "app/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 739, in _r ollback_impl self.engine.dialect.do_rollback(self.connection) File "app/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 22 41, in do_rollback dbapi_connection.rollback() sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2006, 'MySQL server has gone a way') (Background on this error at: http://sqlalche.me/e/e3q8)
The FlaskJob.load_app
has the good intention to reuse the current_app
but IMHO it's making too many assumptions of how the Flask app is structured.
Since RQ is mainly used in backend processes where performance is not the highest priority, I'd like to suggest we call create_app
every time a job is performed. This provides the maximum guarantee on the correctness of the appcontext.