awx awx-manage migrate failed on awx.main.migrations._dab

Please confirm the following

[X] I agree to follow this project's code of conduct.
[X] I have checked the current issues for duplicates.
[X] I understand that AWX is open source software provided for free and that I might not receive a timely response.
[X] I am NOT reporting a (potential) security vulnerability. (These should be emailed to [email protected] instead.)

Bug Summary

awx upgrade from 21.13.0 to 24.6.1 failed. awx-manage migrate failed on awx.main.migrations._dab_rbac step

AWX version

from 21.13.0 to 24.6.1

Select the relevant components

[ ] UI
[ ] UI (tech preview)
[ ] API
[ ] Docs
[ ] Collection
[X] CLI
[ ] Other

Installation method

kubernetes

Modifications

no

Ansible version

No response

Operating system

rhel9.4

Web browser

No response

Steps to reproduce

Upgrade from 21.13.0 to 24.6.1 awx-manage migrate

Expected results

full upgrade passed

Actual results

upgrade failed

Additional information

sh-5.1$ awx-manage migrate 2024-11-13 22:52:24,293 ERROR [-] ansible_base.resource_registry.apps RESOURCE_SERVER is not configured. Reverse sync will not be enabled. 2024-11-13 22:52:24,293 ERROR [-] ansible_base.resource_registry.apps RESOURCE_SERVER is not configured. Reverse sync will not be enabled. 2024-11-13 22:52:24,293 ERROR [-] ansible_base.resource_registry.apps RESOURCE_SERVER is not configured. Reverse sync will not be enabled. Operations to perform: Apply all migrations: auth, conf, contenttypes, dab_rbac, dab_resource_registry, main, oauth2_provider, sessions, sites, social_django, sso Running migrations: Applying main.0192_custom_roles...Adding content type 'main | hostmetricsummarymonthly' Adding content type 'main | inventoryconstructedinventorymembership' Adding content type 'main | receptoraddress' 2024-11-13 22:52:24,963 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Admin pk=RoleDefinition object (241) with 6 permissions 2024-11-13 22:52:24,969 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Inventory Admin pk=RoleDefinition object (242) with 8 permissions 2024-11-13 22:52:24,974 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Use pk=RoleDefinition object (243) with 2 permissions 2024-11-13 22:52:24,979 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Adhoc pk=RoleDefinition object (244) with 2 permissions 2024-11-13 22:52:24,984 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Update pk=RoleDefinition object (245) with 2 permissions 2024-11-13 22:52:24,990 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Team Admin pk=RoleDefinition object (246) with 4 permissions 2024-11-13 22:52:24,994 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Team Member pk=RoleDefinition object (247) with 2 permissions 2024-11-13 22:52:25,000 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Admin pk=RoleDefinition object (248) with 5 permissions 2024-11-13 22:52:25,005 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Project Admin pk=RoleDefinition object (249) with 7 permissions 2024-11-13 22:52:25,011 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Update pk=RoleDefinition object (250) with 2 permissions 2024-11-13 22:52:25,015 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Use pk=RoleDefinition object (251) with 2 permissions 2024-11-13 22:52:25,020 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition JobTemplate Admin pk=RoleDefinition object (252) with 4 permissions 2024-11-13 22:52:25,025 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization JobTemplate Admin pk=RoleDefinition object (253) with 5 permissions 2024-11-13 22:52:25,030 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition JobTemplate Execute pk=RoleDefinition object (254) with 2 permissions 2024-11-13 22:52:25,035 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Credential Admin pk=RoleDefinition object (255) with 4 permissions 2024-11-13 22:52:25,040 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Credential Admin pk=RoleDefinition object (256) with 6 permissions 2024-11-13 22:52:25,045 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Credential Use pk=RoleDefinition object (257) with 2 permissions 2024-11-13 22:52:25,051 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Member pk=RoleDefinition object (258) with 2 permissions 2024-11-13 22:52:25,057 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition InstanceGroup Admin pk=RoleDefinition object (259) with 4 permissions 2024-11-13 22:52:25,062 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition InstanceGroup Use pk=RoleDefinition object (260) with 2 permissions 2024-11-13 22:52:25,067 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition NotificationTemplate Admin pk=RoleDefinition object (261) with 3 permissions 2024-11-13 22:52:24,969 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Inventory Admin pk=RoleDefinition object (242) with 8 permissions 2024-11-13 22:52:24,974 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Use pk=RoleDefinition object (243) with 2 permissions 2024-11-13 22:52:24,979 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Adhoc pk=RoleDefinition object (244) with 2 permissions 2024-11-13 22:52:24,984 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Update pk=RoleDefinition object (245) with 2 permissions 2024-11-13 22:52:24,990 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Team Admin pk=RoleDefinition object (246) with 4 permissions 2024-11-13 22:52:24,994 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Team Member pk=RoleDefinition object (247) with 2 permissions 2024-11-13 22:52:25,000 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Admin pk=RoleDefinition object (248) with 5 permissions 2024-11-13 22:52:25,005 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Project Admin pk=RoleDefinition object (249) with 7 permissions 2024-11-13 22:52:25,011 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Update pk=RoleDefinition object (250) with 2 permissions 2024-11-13 22:52:25,015 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Use pk=RoleDefinition object (251) with 2 permissions 2024-11-13 22:52:25,020 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition JobTemplate Admin pk=RoleDefinition object (252) with 4 permissions 2024-11-13 22:52:25,025 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization JobTemplate Admin pk=RoleDefinition object (253) with 5 permissions 2024-11-13 22:52:25,030 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition JobTemplate Execute pk=RoleDefinition object (254) with 2 permissions 2024-11-13 22:52:25,035 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Credential Admin pk=RoleDefinition object (255) with 4 permissions 2024-11-13 22:52:25,040 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Credential Admin pk=RoleDefinition object (256) with 6 permissions 2024-11-13 22:52:25,045 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Credential Use pk=RoleDefinition object (257) with 2 permissions 2024-11-13 22:52:25,051 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Member pk=RoleDefinition object (258) with 2 permissions 2024-11-13 22:52:25,057 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition InstanceGroup Admin pk=RoleDefinition object (259) with 4 permissions 2024-11-13 22:52:25,062 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition InstanceGroup Use pk=RoleDefinition object (260) with 2 permissions 2024-11-13 22:52:25,067 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition NotificationTemplate Admin pk=RoleDefinition object (261) with 3 permissions 2024-11-13 22:52:25,072 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization NotificationTemplate Admin pk=RoleDefinition object (262) with 5 permissions 2024-11-13 22:52:25,078 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition ExecutionEnvironment Admin pk=RoleDefinition object (263) with 3 permissions 2024-11-13 22:52:25,083 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization ExecutionEnvironment Admin pk=RoleDefinition object (264) with 5 permissions 2024-11-13 22:52:25,089 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition WorkflowJobTemplate Admin pk=RoleDefinition object (265) with 5 permissions 2024-11-13 22:52:25,094 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization WorkflowJobTemplate Admin pk=RoleDefinition object (266) with 7 permissions 2024-11-13 22:52:25,099 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition WorkflowJobTemplate Execute pk=RoleDefinition object (267) with 2 permissions 2024-11-13 22:52:25,104 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition WorkflowJobTemplate Approve pk=RoleDefinition object (268) with 2 permissions 2024-11-13 22:52:25,109 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Admin pk=RoleDefinition object (269) with 46 permissions 2024-11-13 22:52:25,114 INFO [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Audit pk=RoleDefinition object (270) with 10 permissions Traceback (most recent call last): File "/usr/bin/awx-manage", line 8, in sys.exit(manage()) ^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/init.py", line 177, in manage execute_from_command_line(sys.argv) File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/init.py", line 442, in execute_from_command_line utility.execute() File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/init.py", line 436, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv self.execute(*args, **cmd_options) File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/base.py", line 458, in execute output = self.handle(*args, **options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/base.py", line 106, in wrapper res = handle_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/core/management/commands/migrate.py", line 356, in handle post_migrate_state = executor.migrate( ^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate state = self._migrate_all_forwards( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards state = self.apply_migration( ^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration state = migration.apply(state, schema_editor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply operation.database_forwards( File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards self.code(from_state.apps, schema_editor) File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/migrations/_dab_rbac.py", line 196, in migrate_to_new_rbac object = object_cls.objects.get(pk=role.object_id) # WORKAROUND, role.content_object does not work in migrations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/query.py", line 637, in get raise self.model.DoesNotExist( fake.Credential.DoesNotExist: Credential matching query does not exist.

Nov 14 '24 00:11 Lee-Kwang

We could migrate successfully by changing main/migrations/_dab_rbac.py. We put all lines from line# 196 inside try block: try: object = object_cls.objects.get(pk=role.object_id) # WORKAROUND, role.content_object does not work in migrations f = object._meta.get_field(role.role_field) # should be ImplicitRoleField perm_list = get_permissions_for_role(f, children, apps) ..... except object_cls.DoesNotExist: object = None

Above change worked for us.

Dec 06 '24 05:12 Lee-Kwang

This seems like it may be possible if the existing Role objects had a reference to a deleted credential. This is likely something that might be reproducible in a unit test which would lead directly to merging such a fix.

May 15 '25 15:05 AlanCoding