[BUG] Database is locked message during import

[eephyne]

Describe the bug When I import pdf files it sometimes put me a database is locked error (this document is not imported). I think this is happening when I import multiple files at the same time.

To Reproduce import several files in the same time even if I don't think this is happening for everyone.

Expected behavior importing the files without error :P

idea I'm not sure but since my db file is located on a crappy low speed harddrive, my mount point of the container is : /data/documents:/data /data/config/paperlessng:/config

(/data is the mount point of the crappy hdd on my system)

I modified the mount to be a docker volume which is located on a ssd, and it seems there is no more issue. I posted here to inform that it may be a problem with slow hard drive. In case it can be solved or more informative.

From what I looked the db is not in /data, meaning its inside the

Webserver logs

[2022-01-03 19:27:49,249] [ERROR] [paperless.consumer] The following error occured while consuming facture_eau_partie_1_sur_2_02_12_2021.pdf: database is locked

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 84, in _execute

    return self.cursor.execute(sql, params)

  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/sqlite3/base.py", line 423, in execute

    return Database.Cursor.execute(self, query, params)

sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/app/paperless/src/documents/consumer.py", line 287, in try_consume_file

    document = self._store(

  File "/app/paperless/src/documents/consumer.py", line 382, in _store

    document = Document.objects.create(

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/manager.py", line 85, in manager_method

    return getattr(self.get_queryset(), name)(*args, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/query.py", line 453, in create

    obj.save(force_insert=True, using=self.db)

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/base.py", line 726, in save

    self.save_base(using=using, force_insert=force_insert,

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/base.py", line 763, in save_base

    updated = self._save_table(

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/base.py", line 868, in _save_table

    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/base.py", line 906, in _do_insert

    return manager._insert(

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/manager.py", line 85, in manager_method

    return getattr(self.get_queryset(), name)(*args, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/query.py", line 1270, in _insert

    return query.get_compiler(using=using).execute_sql(returning_fields)

  File "/usr/local/lib/python3.8/dist-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql

    cursor.execute(sql, params)

  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 66, in execute

    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)

  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers

    return executor(sql, params, many, context)

  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 84, in _execute

    return self.cursor.execute(sql, params)

  File "/usr/local/lib/python3.8/dist-packages/django/db/utils.py", line 90, in __exit__

    raise dj_exc_value.with_traceback(traceback) from exc_value

  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 84, in _execute

    return self.cursor.execute(sql, params)

  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/sqlite3/base.py", line 423, in execute

    return Database.Cursor.execute(self, query, params)

django.db.utils.OperationalError: database is locked```

**Relevant information**
 - archlinux (runnig on docker using linuxserver.io image.
 - chrome
 - Installation method: docker

[DarrienG]

The linuxserver.io image uses sqlite which does not allow concurrent writes and has to be run in a "job queue" application side or use polling on the lock to see if it works.

If your file eventually uploads, then I wouldn't worry about it (although I suppose the error message could be cleaned up), paperless is just trying to write, and when it fails, it dumps the exception and tries again until it works which is the latter strategy.

If it doesn't eventually upload, then that's definitely a bug. In the meantime if you want this error to go away, moving to a "real" database like MySQL or Postgres will remove the error altogether.

[coolduke666]

got the same issue - any solutions so far (expect migrating to postgres)?

[mikegleasonjr]

Is there a safe path towards a migration from sqlite to an external database?