uploading a large (2+G) raster layer from either QGIS or the web UI time-outs connection

[PeterPetrik]

connection timed out, timeout 300.00

[ERROR] [23] Error handling request /v1/project/push/chunk/181c0e85-5731-4dae-9487-13a9f27e5637/qmqx4lw3atj||Traceback (most recent call last):||  File "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/base_async.py", line 56, in handle||    self.handle_request(listener_name, req, client, addr)||  File "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/ggevent.py", line 160, in handle_request||    addr)||  File "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/base_async.py", line 107, in handle_request||    respiter = self.wsgi(environ, resp.start_response)||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2464, in __call__||    return self.wsgi_app(environ, start_response)||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2450, in wsgi_app||    response = self.handle_exception(e)||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1867, in handle_exception||    reraise(exc_type, exc_value, tb)||  File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 39, in reraise||    raise value||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2447, in wsgi_app||    response = self.full_dispatch_request()||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1952, in full_dispatch_request||    rv = self.handle_user_exception(e)||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1821, in handle_user_exception||    reraise(exc_type, exc_value, tb)||  File "/usr/local/lib/python3.6/dist-packages/flask/_compat.py", line 39, in reraise||    raise value||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1950, in full_dispatch_request||    rv = self.dispatch_request()||  File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1936, in dispatch_request||    return self.view_functions[rule.endpoint](**req.view_args)||  File "/app/src/auth/__init__.py", line 41, in wrapped_func||    if not current_user or not current_user.is_authenticated:||  File "/usr/local/lib/python3.6/dist-packages/werkzeug/local.py", line 329, in __bool__||    return bool(self._get_current_object())||  File "/usr/local/lib/python3.6/dist-packages/werkzeug/local.py", line 307, in _get_current_object||    return self.__local()||  File "/usr/local/lib/python3.6/dist-packages/flask_login/utils.py", line 26, in <lambda>||    current_user = LocalProxy(lambda: _get_user())||  File "/usr/local/lib/python3.6/dist-packages/flask_login/utils.py", line 302, in _get_user||    current_app.login_manager._load_user()||  File "/usr/local/lib/python3.6/dist-packages/flask_login/login_manager.py", line 317, in _load_user||    return self.reload_user()||  File "/usr/local/lib/python3.6/dist-packages/flask_login/login_manager.py", line 279, in reload_user||    user = self.user_callback(user_id)||  File "/app/src/auth/__init__.py", line 97, in load_user||    return User.query.get(user_id)||  File "<string>", line 2, in get||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/deprecations.py", line 390, in warned||    return fn(*args, **kwargs)||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 933, in get||    return self._get_impl(ident, loading.load_on_pk_identity)||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 945, in _get_impl||    execution_options=self._execution_options,||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 2800, in _get_impl||    load_options=load_options,||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/loading.py", line 535, in load_on_pk_identity||    bind_arguments=bind_arguments,||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 1669, in execute||    conn = self._connection_for_bind(bind, close_with_result=True)||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 1520, in _connection_for_bind||    engine, execution_options||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 747, in _connection_for_bind||    conn = bind.connect()||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 3096, in connect||    return self._connection_cls(self, close_with_result=close_with_result)||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 92, in __init__||    else engine.raw_connection()||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 3175, in raw_connection||    return self._wrap_pool_connect(self.pool.connect, _connection)||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 3142, in _wrap_pool_connect||    return fn()||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 301, in connect||    return _ConnectionFairy._checkout(self)||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 761, in _checkout||    fairy = _ConnectionRecord.checkout(pool)||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/base.py", line 419, in checkout||    rec = pool._do_get()||  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/pool/impl.py", line 137, in _do_get||    code="3o7r",||sqlalchemy.exc.TimeoutError: QueuePool limit of size 2 overflow 10 reached, connection timed out, timeout 300.00 (Background on this error at

[ppetru]

Gathered some info when this happens through the QGIS plugin.

Setup: project already synched, new large (2.3G) raster layer added to the project directory and sync initiated through the QGIS plugin.

Server log: [ACCESS] 192.168.1.1 GET /v1/project/ppetru/Alo survey since=v4 HTTP/1.1 200 9156 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 53616 <29> [ACCESS] 192.168.1.1 GET /v1/project/ppetru/Alo survey since=v4 HTTP/1.1 200 9156 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 33721 <29> [ACCESS] 192.168.1.1 GET /v1/project/ppetru/Alo survey HTTP/1.1 200 8274 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 28624 <29> [ACCESS] 192.168.1.1 GET /v1/project/ppetru/Alo survey HTTP/1.1 200 8274 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 25385 <29> [ACCESS] 192.168.1.1 GET /v1/user/ppetru HTTP/1.1 200 401 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 79382 <29> [ACCESS] 192.168.1.1 POST /v1/project/push/ppetru/Alo survey HTTP/1.1 200 60 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 870456 <29> [ACCESS] 192.168.1.1 POST /v1/project/push/chunk/a861def0-0e77-4ed1-b6d0-978e338bc8be/3f974455-a47f-43a8-b383-450615c59e88 HTTP/1.1 200 82 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 697662 <28> [ACCESS] 192.168.1.1 POST /v1/project/push/chunk/a861def0-0e77-4ed1-b6d0-978e338bc8be/7bd20344-5e60-459f-831d-e8ea6133c1f7 HTTP/1.1 200 82 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 428456 <28> [ACCESS] 192.168.1.1 POST /v1/project/push/chunk/a861def0-0e77-4ed1-b6d0-978e338bc8be/26e78023-ccb9-4cd2-8932-1c49936efa2f HTTP/1.1 200 82 - "Python-client/0.6.4 Plugin/2021.4.2 QGIS/3.20.3 (Windows/10)" 1574671 <29>

[lots more like the previous 3, with the chunk ID and size varying]

The plugin progress bar reaches 100%, then hangs. Server log:

[CRITICAL] [12] WORKER TIMEOUT (pid:29) /usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/strategies.py:551: SAWarning: LRU cache size alert for loader strategy: Upload.project util.warn("LRU cache size alert for loader strategy: %s" % self) [INFO] [29] worker received SIGABRT signal [INFO] [30] Booting worker with pid: 30 [CRITICAL] [12] WORKER TIMEOUT (pid:28) /usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/strategies.py:551: SAWarning: LRU cache size alert for loader strategy: Upload.project util.warn("LRU cache size alert for loader strategy: %s" % self) [INFO] [28] worker received SIGABRT signal [INFO] [28]

Thread: (140704632547136)

File: "/usr/local/lib/python3.6/dist-packages/gevent/baseserver.py", line 34, in _handle_and_close_when_done return handle(*args_tuple) File: "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/ggevent.py", line 155, in handle super(GeventWorker, self).handle(listener, client, addr) File: "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/base_async.py", line 56, in handle self.handle_request(listener_name, req, client, addr) File: "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/ggevent.py", line 160, in handle_request addr) File: "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/base_async.py", line 107, in handle_request respiter = self.wsgi(environ, resp.start_response) File: "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2464, in call return self.wsgi_app(environ, start_response) File: "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File: "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File: "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File: "/usr/local/lib/python3.6/dist-packages/connexion/decorators/decorator.py", line 48, in wrapper response = function(request) File: "/usr/local/lib/python3.6/dist-packages/connexion/decorators/uri_parsing.py", line 144, in wrapper response = function(request) File: "/usr/local/lib/python3.6/dist-packages/connexion/decorators/validation.py", line 384, in wrapper return function(request) File: "/usr/local/lib/python3.6/dist-packages/connexion/decorators/parameter.py", line 121, in wrapper return function(**kwargs) File: "/app/src/auth/init.py", line 47, in wrapped_func return f(*args, **kwargs) File: "/app/src/controllers/project_controller.py", line 800, in push_cancel db.session.commit() File: "", line 2, in commit File: "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 1423, in commit self._transaction.commit(_to_root=self.future) File: "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/session.py", line 836, in commit trans.commit() File: "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2266, in commit self._do_commit() File: "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2441, in _do_commit self._connection_commit_impl() File: "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 2412, in _connection_commit_impl self.connection._commit_impl() File: "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 929, in _commit_impl self.engine.dialect.do_commit(self.connection) File: "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 669, in do_commit dbapi_connection.commit() File: "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/base.py", line 195, in handle_abort self.cfg.worker_abort(self) File: "config.py", line 68, in worker_abort for filename, lineno, name, line in traceback.extract_stack(stack): [WARNING] [28] Worker graceful timeout (pid:28) [INFO] [28] Worker exiting (pid: 28) [INFO] [31] Booting worker with pid: 31

The plugin shows: "Client error: Bad Gateway".

[varmar05]

There were multiple fixes for sync process done in 2023.2.0, so this should not be an issue anymore. However download of large zip project archives was limited to certain size (around 1 GB)