Duplicate mirror in AlmaLinux 8 causes exception

[RicardoJeronimo]

Hey!

After adding some AlmaLinux 8.8 and 8.10 to my Patchman instance, I'm getting the following error on patchman -a (tested on both v2 and v3):

Repository 522 : AlmaLinux 8.10 - AppStream x86_64
Found mirrorlist - https://mirrors.almalinux.org/mirrorlist/8.10/appstream
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
repos.models.Mirror.DoesNotExist: Mirror matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 239, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.IntegrityError: (1062, "Duplicate entry 'http://mirrors.ptisp.pt/almalinux/8.10/AppStream/x86_64/os/' for key 'repos_mirror.url'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/patchman", line 705, in <module>
    main()
  File "/usr/bin/patchman", line 699, in main
    showhelp = process_args(args)
               ^^^^^^^^^^^^^^^^^^
  File "/usr/bin/patchman", line 665, in process_args
    refresh_repos(args.repo, args.force)
  File "/usr/bin/patchman", line 122, in refresh_repos
    repo.refresh(force)
  File "/usr/lib/python3/dist-packages/repos/models.py", line 86, in refresh
    refresh_rpm_repo(self)
  File "/usr/lib/python3/dist-packages/repos/utils.py", line 687, in refresh_rpm_repo
    check_for_mirrorlists(repo)
  File "/usr/lib/python3/dist-packages/repos/utils.py", line 271, in check_for_mirrorlists
    add_mirrors_from_urls(repo, mirror_urls)
  File "/usr/lib/python3/dist-packages/repos/utils.py", line 253, in add_mirrors_from_urls
    m, c = Mirror.objects.get_or_create(repo=repo, url=mirror_url)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 588, in get_or_create
    return self.create(**params), True
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
              ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/lib/python3/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 73, in execute
    return self.cursor.execute(query, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
          ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 239, in query
    _mysql.connection.query(self, query)
django.db.utils.IntegrityError: (1062, "Duplicate entry 'http://mirrors.ptisp.pt/almalinux/8.10/AppStream/x86_64/os/' for key 'repos_mirror.url'")

After digging a little bit, I noticed that all mirrorlists for AlmaLinux 8 use 8.10 URLs, causing the error above: AlmaLinux 8: https://mirrors.almalinux.org/mirrorlist/8/appstream AlmaLinux 8.10: https://mirrors.almalinux.org/mirrorlist/8.10/appstream

In my case, the v8 mirror list is used by (almost) all AlmaLinux 8.8 servers:

While the v8.10 mirrolist is used by all AlmaLinux 8.10 servers (including some AlmaLinux 8.8 servers as well):

Is this something that can be fixed? I tried to disable the duplicate mirrors without success.

Thank you!