Perform project updates and inventory updates on correct branch

[willthames]

SUMMARY

Ensure that job template's branch is taken into account when performing inventory and source updates

Without this change, scm branch isn't passed to the update jobs and it ends up using HEAD, which is only ever accidentally what is wanted.

This has no effect on the actual job being run, which does run against the correct branch (but potentially with an incorrect inventory).

ISSUE TYPE

• Bug, Docs Fix or other nominal change

COMPONENT NAME

• API

AWX VERSION

awx: 22.1.1.dev136+gb310f4fd56.d20230609

ADDITIONAL INFORMATION

Reproduction

• Create a project with an inventory source and job template based on an SCM repository. Project and job template should have a default branch.
• Update HEAD of the SCM repository to something different to the default branch
• Run the project - the project will run against the correct branch but the Source Control Update and Inventory Sync will run against HEAD.

[AlanCoding]

I thought the issue here was resolved by the merge of https://github.com/ansible/awx/pull/12073 by @fosterseth

I want to understand a little bit about what that didn't do which is now being requested here. I'm sure there are open issues on this subject.

[willthames]

@AlanCoding I have now properly documented my findings with a minimal reproducible test suite.

See https://github.com/willthames/awx-project-update-test-suite for the details.

Note that with the change in this PR, Test 1 still fails (my workaround for that is to set project default branch), Tests 3 and 4 both pass.

Removing lines 282 and 283 as you suggest causes Test 4 to fail - it runs using the override branch playbook but against the job-default inventory (I only retried the second part of the test suite, Test 3 does succeed)

[AlanCoding]

https://github.com/ansible/awx/pull/13644 was the recent patch I was trying to think of. I didn't understand your test case fully.

[Aohzan]

#13644 was the recent patch I was trying to think of. I didn't understand your test case fully.

hello, we have the same need: we want to dynamically choose the inventory source branch from the job template task and not "statically" from the inventory source. It will allowed multiple user to run playbooks with an inventory from their branch

[pierre-cegid]

That is exactly what we need, thank you very much @willthames ! We are looking to propagate the scm branch set in awx template to awx project and awx inventory sources.

[willthames]

@AlanCoding can I rerequest another look at this one?

I've been using this version of task_manager.py in our AWX for months now (occasionally rebasing it during AWX upgrades) and I'm very happy with how it works - would be good to get it merged in!

[willthames]

Ah, I was looking at my wrong source branch, it seems that task manager has been refactored since I raised this PR and that the remaining changes I have may no longer be required - I'll validate that and either update this PR or close it

[willthames]

I've added some tests to AWX to validate the use of scm_branch with job templates and tasks and updated the code to adjust for the very significant refactor of the underlying workflows.

I'm happy that the tests pass after passing scm_branch to the inventory update and project update tasks

There are some changes to allow for new status that there might be a better way to do (I couldn't get the inventory update to ever fire, even for the known good test case, without that change)

[dmzoneill]

@willthames Thanks for your continued work on this. I'm moving it to draft for the time being to make it clearer to developers what is ready for merging. Please once the undraft once it's ready.

[willthames]

Pretty sure the failing test looks like the RPM repo was temporarily down, I don't have permissions to retry a single test though

[fosterseth]

@willthames thanks for the PR, reviewing but having trouble understanding the root issue

1. I setup a project with branch set to foo
2. I setup a job template based on that project, also with branch set to foo
3. I setup an inventory source based on that project, with nothing set in the branch field
4. importantly update on launch enabled for the project and inventory source
5. Go to github and change my inventory.ini file with a different host (127.0.0.32)

when I launch job I get the following updates spawned,

the job runs against the right host

[willthames]

Hi @fosterseth , the accompanying tests in this PR should show the behaviour, as I wrote them to fail without this change.

The problem is when using a branch that isn't the default branch for the project/job template

• Create a project with default branch main
• Create a job template with default branch main
• Create a branch with an inventory update (you could use foo if it only had the change above) and a task change
• Run a job using foo branch
• It will probably use main for inventory and foo for the tasks - you can validate this even without looking at the output by going to the underlying jobs and seeing what revision the jobs ran under, but making the outputs change make it a little more obvious

Let me know if you need any further info.