patchman icon indicating copy to clipboard operation
patchman copied to clipboard

Published 20 hours ago • verified publisher icon furlongm

django.db.utils.OperationalError: database is locked

Open jaydeepch15 opened this issue 2 years ago • 2 comments

Getting below error on server while running patchman -a command

Traceback (most recent call last):  26%|###############                                          |ETA:  0:00:34
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 538, in get_or_create
    return self.get(**kwargs), False
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 406, in get
    raise self.model.DoesNotExist(
packages.models.DoesNotExist: Package matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/sqlite3/base.py", line 383, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/patchman", line 686, in <module>
    main()
  File "/usr/bin/patchman", line 680, in main
    showhelp = process_args(args)
  File "/usr/bin/patchman", line 646, in process_args
    refresh_repos(args.repo, args.force)
  File "/usr/bin/patchman", line 121, in refresh_repos
    repo.refresh(force)
  File "/usr/lib/python3/dist-packages/repos/models.py", line 86, in refresh
    refresh_rpm_repo(self)
  File "/usr/lib/python3/dist-packages/repos/utils.py", line 616, in refresh_rpm_repo
    refresh_yum_repo(mirror, data, mirror_url, ts)
  File "/usr/lib/python3/dist-packages/repos/utils.py", line 481, in refresh_yum_repo
    update_mirror_packages(mirror, packages)
  File "/usr/lib/python3/dist-packages/repos/utils.py", line 116, in update_mirror_packages
    p, c = all_packages.get_or_create(name=package_id,
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 541, in get_or_create
    return self._create_object_from_params(kwargs, params)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 575, in _create_object_from_params
    obj = self.create(**params)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 422, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 740, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 777, in save_base
    updated = self._save_table(
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 870, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 907, in _do_insert
    return manager._insert([self], fields=fields, return_id=update_pk,
  File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1186, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1375, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/sqlite3/base.py", line 383, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: database is locked

jaydeepch15 avatar Apr 18 '22 08:04 jaydeepch15

I tried with deleting Errata from django admin then run patchman -d

root@awslpatchman:/var/lib/patchman/db# patchman -d
Removing 731 orphaned Packages:    100%|##################################################################################################################################################|Time: 0:00:06
Removing 30 unused Package names:  100%|##################################################################################################################################################|Time: 0:00:00
No orphaned Package Architectures found.
Removing 2 orphaned M Arches:      100%|##################################################################################################################################################|Time: 0:00:00
No Repositories with zero Mirrors found.

again ran patchman -e

root@awslpatchman:/var/lib/patchman/db# patchman -e
Downloading Errata Checksum:       100%|##################################################################################################################################################|Time: 0:00:00
Downloading CentOS Errata:         100%|##################################################################################################################################################|Time: 0:00:00
Traceback (most recent call last):   2%|####                                                                                                                                              |ETA:  0:18:35
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/sqlite3/base.py", line 383, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/patchman", line 686, in <module>
    main()
  File "/usr/bin/patchman", line 680, in main
    showhelp = process_args(args)
  File "/usr/bin/patchman", line 663, in process_args
    update_errata(args.force)
  File "/usr/lib/python3/dist-packages/packages/utils.py", line 89, in update_errata
    parse_errata(bunzip2(data), force)
  File "/usr/lib/python3/dist-packages/packages/utils.py", line 126, in parse_errata
    e = parse_errata_tag(child.tag, child.attrib, force)
  File "/usr/lib/python3/dist-packages/packages/utils.py", line 152, in parse_errata_tag
    add_erratum_refs(e, references)
  File "/usr/lib/python3/dist-packages/packages/utils.py", line 230, in add_erratum_refs
    e.references.add(er)
  File "/usr/lib/python3/dist-packages/django/db/models/fields/related_descriptors.py", line 936, in add
    self._add_items(
  File "/usr/lib/python3/dist-packages/django/db/models/fields/related_descriptors.py", line 1080, in _add_items
    self.through._default_manager.using(db).bulk_create([
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 474, in bulk_create
    ids = self._batched_insert(objs_without_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1211, in _batched_insert
    self._insert(item, fields=fields, using=self.db, ignore_conflicts=ignore_conflicts)
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1186, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1375, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/sqlite3/base.py", line 383, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: database is locked

even tried with fuser -k patchman.db Everytime it ends with same error.

jjchaudhari avatar Apr 19 '22 10:04 jjchaudhari

To check I switched from sqlite to mysql when ran patchman -a get below error

Removing 1198 orphaned Packages:   100%|##################################################################################################################################################|Time: 0:02:05
Removing 931 unused Package names: 100%|##################################################################################################################################################|Time: 0:00:58
Removing 2 orphaned P Arches:      100%|##################################################################################################################################################|Time: 0:00:00
No orphaned Machine Architectures found.
No Repositories with zero Mirrors found.
Removing 4 unused tagged items     100%|##################################################################################################################################################|Time: 0:00:00
Refreshing metadata for all Repos

Repository 1 : Amazon Linux 2 core repository x86_64
Downloading repo info (1/2):       100%|##################################################################################################################################################|Time: 0:00:00
Found yum rpm repo - http://amazonlinux.us-east-1.amazonaws.com/2/core/2.0/x86_64/37eaca2854cfb48e16ced2e2a46fb7e92869f36c054be627be581970bc437a42/repodata/repomd.xml
Downloading repo info (2/2):       100%|##################################################################################################################################################|Time: 0:01:27
Killed

I tried to increase innodb_lock_wait_timeout but it also didn't worked

mysql> show variables like 'innodb_lock_wait_timeout';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| innodb_lock_wait_timeout | 200   |
+--------------------------+-------+
1 row in set (0.01 sec)

mysql>
mysql> SHOW GLOBAL VARIABLES LIKE 'innodb_lock_wait_timeout';
+--------------------------+-------+
| Variable_name            | Value |
+--------------------------+-------+
| innodb_lock_wait_timeout | 200   |
+--------------------------+-------+
1 row in set (0.00 sec)

Everything seems running fine on instance

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      2153/mysqld
tcp        0      0 0.0.0.0:1514            0.0.0.0:*               LISTEN      633/al-slc.current
tcp        0      0 127.0.0.1:587           0.0.0.0:*               LISTEN      820/sendmail: MTA:
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      708/redis-server 12
tcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTEN      499/memcached
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      439/systemd-resolve
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      887/sshd: /usr/sbin
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      820/sendmail: MTA:
tcp        0      0 127.0.0.1:33060         0.0.0.0:*               LISTEN      2153/mysqld
tcp6       0      0 ::1:6379                :::*                    LISTEN      708/redis-server 12
tcp6       0      0 :::80                   :::*                    LISTEN      12567/apache2
tcp6       0      0 :::22                   :::*                    LISTEN      887/sshd: /usr/sbin

jjchaudhari avatar Apr 19 '22 12:04 jjchaudhari

Just checking - is DEBUG=True set in your local settings file?

furlongm avatar Dec 23 '22 19:12 furlongm

Seems to be a memory issue. Feel free to reopen if you have more details.

furlongm avatar May 02 '23 16:05 furlongm