meeb
database is locked errors in the docker logs
Getting the following in the logs and can see multiple sync tasks are stuck in the interface.. I've tried a reset tasks but they dont get removed.
This started happening since I did docker compose pull recently.
` The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/huey/api.py", line 291, in _emit self._signal.send(signal, task, *args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/huey/signals.py", line 43, in send receiver(signal, task, *args, **kwargs) File "/app/common/huey.py", line 377, in historical_task th, created = TaskHistory.objects.get_or_create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/manager.py", line 87, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 951, in get_or_create with transaction.atomic(using=self.db): File "/usr/local/lib/python3.11/dist-packages/django/db/transaction.py", line 216, in enter connection.set_autocommit( File "/usr/local/lib/python3.11/dist-packages/django/db/backends/base/base.py", line 482, in set_autocommit self._start_transaction_under_autocommit() File "/usr/local/lib/python3.11/dist-packages/django/db/backends/sqlite3/base.py", line 332, in _start_transaction_under_autocommit self.cursor().execute(f"BEGIN {self.transaction_mode}") File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 79, in execute return self._execute_with_wrappers( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers return executor(sql, params, many, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 100, in _execute with self.db.wrap_database_errors: File "/usr/local/lib/python3.11/dist-packages/django/db/utils.py", line 91, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 103, in _execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/sqlite3/base.py", line 356, in execute return super().execute(query) ^^^^^^^^^^^^^^^^^^^^^^ django.db.utils.OperationalError: database is locked [2025-07-16 08:48:00,141] ERROR:huey:Worker-2:Error occurred sending signal "enqueued" Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 946, in get_or_create return self.get(**kwargs), False ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 633, in get raise self.model.DoesNotExist( common.models.tasks.TaskHistory.DoesNotExist: TaskHistory matching query does not exist.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 103, in _execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/sqlite3/base.py", line 356, in execute return super().execute(query) ^^^^^^^^^^^^^^^^^^^^^^ sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/huey/api.py", line 291, in _emit self._signal.send(signal, task, *args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/huey/signals.py", line 43, in send receiver(signal, task, *args, **kwargs) File "/app/common/huey.py", line 377, in historical_task th, created = TaskHistory.objects.get_or_create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/manager.py", line 87, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 951, in get_or_create with transaction.atomic(using=self.db): File "/usr/local/lib/python3.11/dist-packages/django/db/transaction.py", line 216, in enter connection.set_autocommit( File "/usr/local/lib/python3.11/dist-packages/django/db/backends/base/base.py", line 482, in set_autocommit self._start_transaction_under_autocommit() File "/usr/local/lib/python3.11/dist-packages/django/db/backends/sqlite3/base.py", line 332, in _start_transaction_under_autocommit self.cursor().execute(f"BEGIN {self.transaction_mode}") File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 79, in execute return self._execute_with_wrappers( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers return executor(sql, params, many, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 100, in _execute with self.db.wrap_database_errors: File "/usr/local/lib/python3.11/dist-packages/django/db/utils.py", line 91, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 103, in _execute return self.cursor.execute(sql) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/sqlite3/base.py", line 356, in execute return super().execute(query) ^^^^^^^^^^^^^^^^^^^^^^ django.db.utils.OperationalError: database is locked [2025-07-16 08:49:18,010] ERROR:huey:Worker-2:Error occurred sending signal "enqueued" Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 105, in _execute return self.cursor.execute(sql, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/sqlite3/base.py", line 360, in execute return super().execute(query, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/huey/api.py", line 291, in _emit self._signal.send(signal, task, *args, **kwargs) File "/usr/local/lib/python3.11/dist-packages/huey/signals.py", line 43, in send receiver(signal, task, *args, **kwargs) File "/app/common/huey.py", line 416, in historical_task th.save() File "/app/common/models/tasks.py", line 121, in save return super().save(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 902, in save self.save_base( File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 1008, in save_base updated = self._save_table( ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 1138, in _save_table updated = self._do_update( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 1203, in _do_update return filtered._update(values) > 0 ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 1286, in _update return query.get_compiler(self.db).execute_sql(ROW_COUNT) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/sql/compiler.py", line 2060, in execute_sql row_count = super().execute_sql(result_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/models/sql/compiler.py", line 1623, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 79, in execute return self._execute_with_wrappers( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers return executor(sql, params, many, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 100, in _execute with self.db.wrap_database_errors: File "/usr/local/lib/python3.11/dist-packages/django/db/utils.py", line 91, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 105, in _execute return self.cursor.execute(sql, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/django/db/backends/sqlite3/base.py", line 360, in execute return super().execute(query, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ django.db.utils.OperationalError: database is locked `
The tasks page reads from the TaskHistory model.
Resetting the tasks does work. It was one of the first things that I made work when starting on the new tasks system.
The tasks page doesn't directly query the tasks system, it reads the history records instead.
That's why it looks like there are still tasks running.
Task history entries older than the maximum running time are excluded and records older than 7 days are removed anyway.
Your tasks page will eventually catch up to the current state after resetting tasks.
The "database is locked" errors are fairly common with SQLite. You can try using the write-ahead-log mode, or increasing the timeout.
However, it's best to switch to PostgreSQL for better concurrency support.
Thanks for the clarification about the tasks (y)
Still getting the database is locked errors which I've never seen before.
I tried to switch to PostgreSQL previously but couldn't seem to get it to work. I'm not an expert with docker compose!
What steps have you taken?
Have you run the VACUUM^vacuum command recently?
https://github.com/meeb/tubesync/blob/802716047c23df7248c3970e2eda35e9995bd6ca/tubesync/tubesync/local_settings.py.container#L47-L72
Have you increased the timeout in the local_settings.py file?
https://github.com/meeb/tubesync/blob/802716047c23df7248c3970e2eda35e9995bd6ca/tubesync/tubesync/local_settings.py.container#L52
Have you changed the journal_mode to WAL^wal for your SQLite database?
https://github.com/meeb/tubesync/blob/802716047c23df7248c3970e2eda35e9995bd6ca/tubesync/tubesync/local_settings.py.container#L56
Perhaps it is time to try again with PostgreSQL?