Intended job Git sync doesn't refresh config contexts in shared repo

[matt852]

Environment

• Python version: 3.8.6
• Nautobot version: 1.5.12
• nautobot-golden-config version: 1.3.1

Steps to Reproduce

1. Create a generic Cisco NXOS device in Nautobot
2. Create a Git Repo called nautobot-gc-files with 2 folders in it: "config_contexts" and "templates". Under provides, set it to both config contexts and jinja templates
3. Create templates/cisco_nxos.j2 (file contents below)
4. Create config_contexts/all.yml (file contents below)
5. In Nautobot, sync the Git repo
6. In Nautobot, run the job Execute All Golden Configuration Jobs - Single Device
7. View results. The device should have 1 AAA server under its config_context tab, and 1 feature in the template/intended config
8. In the Git repo, remove the AAA server in config_contexts/all.yml, and remove the feature command in templates/cisco_nxos.j2
9. Rerun the job Execute All Golden Configuration Jobs - Single Device. Do NOT manually sync the Git repo
10. View results. The feature has been deleted in the intended config, but the AAA server remains
11. Manually sync the repo, then rerun the job Execute All Golden Configuration Jobs - Single Device
12. View results. AAA server has been deleted

Files

templates/cisco_nxos.j2

feature vtp

config_contexts/all.yml

---
_metadata:
  name: "all"
  weight: 500
  description: "Global Definitions"
  is_active: true
aaa:
  - ip: 1.1.1.1

Expected Behavior

When I run the Golden Config job Execute All Golden Configuration Jobs - Single Device, I expect the repo to sync and both the Jinja2 templates as well as the Config Context data to refresh.

Observed Behavior

The job appears to sync the repository correctly, but only refreshes the template data and not the config context data. If I manually sync the Git repository, the data is updated correctly on the next job run.

This does not seem to be an issue when the config contexts and jinja2 templates are configured in separate repos.

Note the same behavior applies with these jobs:

• Execute All Golden Configuration Jobs - Single Device
• Execute All Golden Configuration Jobs - Multiple Device
• Generated Intended Configurations

[itdependsnetworks]

If I understand correctly, you are saying that the job does not automatically sync the git repos, such as data or templates? If so, this is to be categorized as "works as intended" https://docs.nautobot.com/projects/golden-config/en/latest/user/app_faq/#why-arent-configurations-generated-or-compliance-generated-real-time

[matt852]

The jobs sync the repo but only refreshes Template data, even when both Config Context data and Template data are in the same repo.

[jeffkala]

We talked about this a bit this morning and may consider adding this as a settings option on whether or not to automatically sync config context data as part of the job. This will allow it to remain backwards compat., but also gives flexibility for those cases where multiple teams/ppl maybe updating config-context data in the repo which an auto sync may not yield the expected results.

[matt852]

I think that works, but I think I might be making this sound a bit more complex than I mean to, or I am simply misunderstanding.

Here's a screenshot of running the "Intended Configs" job.

The job refreshes the Templates repository and the templates from the repo. However it does NOT refresh the config contexts repo, nor does it update the data from the config contexts repo. This repo is flagged for both config_context data and jinja2 templates.

Since the repo is serving a dual purpose, shouldn't this job either refresh both or none at all?

[jeffkala]

We're targeting off of repo + "provides". So it's working as expected. I don't believe you're misunderstanding anything here.

[netopsengineer]

I also find this default behavior to be troublesome for my teams, in my case though we are using split repos, one for templates, one for contexts, and multiple external teams update the config contexts and templates, for which they own and maintain, and this leads to a lot of "my compliance isn't updating" messages to the Application SME's, because they forget that they have to manually go refresh the config context repo every time, before running compliance, but not for jinja template updates. Perhaps, the suggestion of a setting mentioned earlier to retain backwards compatibility would be a good option.

[jdrew82]

This has been addressed in #506 and #543.