project updates due to scm_branch do not show in UI list under Jobs

[bartowl]

Please confirm the following

• [X] I agree to follow this project's code of conduct.
• [X] I have checked the current issues for duplicates.
• [X] I understand that AWX is open source software provided for free and that I might not receive a timely response.

Bug Summary

When project allows to override branch, and job template is started with a particular scm_branch, this triggers few things:

1. project update with project default branch (if project set with update revision on launch) - this seems to be obsole because of following point
2. an additional project update is started (new ID) against the given scm_branch
3. at this moment the playbook is executed

The problem is, that in UI under Jobs Source Control Update for point 1 is visible with example ID of 100, but the project update from point 2, though it got registered under ID 101 is not displayed regardless of filter settings. Displaying Source Control Update with ID 100 and then changing URL to ID 101 shows the project update resulting from scm_branch.

First of all, project update with generic branch does not make much sense if scm_branch is given, as it just wastes time and will be anyway overwritten. Second of all - source control updates resulting from custom scm_branch are not shown in the UI at all. In API this works allright, when asking via UI for a given ID this also works. It just does not get listed.

AWX version

21.1.0

Select the relevant components

• [X] UI
• [ ] API
• [ ] Docs
• [ ] Collection
• [ ] CLI
• [ ] Other

Installation method

kubernetes

Modifications

no

Ansible version

2.12.5.post0

Operating system

CensOS Stream 8 (awx-operator default)

Web browser

Chrome

Steps to reproduce

1. create a project with a simple playbook as git repository.
2. In this repository create 2 branches - main and other, both should include site.yml.
3. set project to branch main, select following options:
   • delete
   • update revision on launch
   • allow branch override
4. create job template for this project with site.yml and other as Source Control Branch
5. execute this job template

Expected results

Under Jobs in UI should show up ideally only 2 entries: ID:n - playbook run (waiting for following 1 to complete) ID:n+1 - Source Control Update (against branch other)

Optionally (but not performant) all currently present IDs should show up, which is: ID:n - playbook run (waiting for following 2 to complete) ID:n+1 - Source Control Update (against branch main) ID:n+2 - Source Control Update (against branch other)

Actual results

Currently under jobs following shows up: ID:n - playbook run (waiting for following 2 to complete) ID:n+1 - Source Control Update (against branch main)

ID:n+2 - Source Control Update (against branch other) altough present is not being shown in the list. If the branch has some errornous requirements.yml which causes this task to fail, you need to manually alter URL to show it (/#/jobs/project/(n+2)/output)

Additional information

This is not a major problem, but just a bit annoying when debuggin problems with requirements.yml on branch. IT also takes double the time as it twice downloads the repository and all roles/collections from all requirement.yml files...

[fosterseth]

Thanks for opening the issue, would you mind providing some screenshots of the UI that demonstrates the issue?

[bartowl]

sure - those are subsequent runs of a job template, that has Source Control Branch specified: As you see by the ID sequences, the intermediate project updates are not being shown. When I target particular URL I see that job, but it does not show up in the list:

[azrdev]

I also see these duplicate project syncs on AWX 21.1.

~~Moreover, the playbook run seems to use the synced code from the wrong one, i.e. without the overridden branch: I'm adding a requirements.yml to use the community.general collection on my feature branch, which is correctly installed (only) in the hidden project sync, and the playbook fails with couldn't resolve module/action.~~

Edit: not sure if that's really the issue, I'm not seeing my collection in the playbook even when adding the requirements.yml to the default branch (and thusly seeing it installed in both project syncs)

[mabashian]

Sync jobs launched in this fashion have always been filtered out of the jobs list: https://github.com/ansible/awx/blob/devel/awx/ui/src/components/JobList/JobList.js#L40

We could look at making changes to the interface that would allow a user to opt out of this filter if they wanted to see these types of sync jobs in the list. I think for some users this would make the list extremely noisy so i'm hesitant to remove the filter.

[azrdev]

1. this would never add more than one extra sync job, correct? I would not consider that too noisy compared to the uselessness of sync jobs in general (if nothing is broken) which can be simply filtered away by looking only at job_type=playbook run
2. the real fix here would be to get rid of the sync job with the default branch, or is there a reason it's needed for?

[bartowl]

I would way if you can display details of both kind of sync jobs via proper URL they should show up in the list. I agree for a filter of job typoe - sync job vs playbook vs worhflow etc, but not having a way to list a job that may fail is a bit problematic. I mean for those who know it is not a problem to construct URL, but this is a bit messy... For me you can set also default filter to hide those sync jobs, but there should be a way to display all of them. And due to the way AWX works I do understand, that unning custom branch needs to perform a checkout and re-check galaxy modules/roles so there is a need for this extra sync job. And the default sync job must not run with each job execution, but when it does it is good to see it in the list, even if just for knowing that it might not need be running for each job run...