Downgrade cloud-init to 23.3 via epoch

[rlmenge]

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

• [x] The toolchain has been rebuilt successfully (or no changes were made to it)
• [x] The toolchain/worker package manifests are up-to-date
• [x] Any updated packages successfully build (or no packages were changed)
• [x] Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
• [x] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
• [x] All package sources are available
• [x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
• [x] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
• [x] All source files have up-to-date hashes in the *.signatures.json files
• [x] sudo make go-tidy-all and sudo make go-test-coverage pass
• [x] Documentation has been updated to match any changes to the build system
• [x] Ready to merge

---

Summary

Up epoch to downgrade cloud-init to 23.3. cloud-init 23.4 introduced a change which requires rfc3442 support. Currently, Mariner's dhclient does not fully support rfc3442. This leads to failure in establishing global routing. Downgrade cloud-init to remove rfc3442 requirement.

Note that the diff in version of 23.3 and not 23.3.3 as mentioned in source url is intentional

Change Log

• Downgrade to 23.3 using epoch 1
• Add back overrideDatasourceDetection.patch
• Remove ci-Pin-pytest-8.0.0.patch as pytest is pinned to pytest<=7.3.1
• Remove Retain-exit-code-in-cloud-init-status-for-recoverabl.patch

Does this affect the toolchain?

NO

Associated issues

• https://microsoft.visualstudio.com/OS/_workitems/edit/49661506

Test Methodology

• Pipeline build id w/ tests (100% pass): https://dev.azure.com/mariner-org/mariner/_build/results?buildId=545390&view=results
• Built locally within toolkit
• Built locally within container w/ tests (100% pass:) =========== 4861 passed, 8 skipped, 1 xfailed, 3 warnings in 55.74s ============"
• Full build: https://dev.azure.com/mariner-org/mariner/_build/results?buildId=545392&view=results
• Penguinator build with no fails: Pipelines - Run 20240404.5 (azure.com)
• Fasttrack run of tests: https://dev.azure.com/mariner-org/mariner/_build/results?buildId=545397&view=results

[dmcilvaney]

Also done here: https://github.com/microsoft/azurelinux/pull/8705#issuecomment-2038433499, results the same.

Assessment of epoch handling inside our build tools:

Setup

I created a test environment with the following setup:

• dedicated ./SPECS-TEST folder containing:
  • a.spec: dummy package with BuildRequires: cloud-init.
• prepopulated ./out/RPMS
  • cloud-init-23.3-1.cm2.noarch.rpm
  • cloud-init-23.4.1-3.cm2.noarch.rpm
  • cloud-init-azure-kvp-23.3-1.cm2.noarch.rpm
  • cloud-init-azure-kvp-23.4.1-3.cm2.noarch.rpm

Test 1

Running sudo make build-packages SPECS_DIR=../SPECS-TEST PACKAGE_REBUILD_LIST=a shows that the graph fetcher selects 1:23.3. This represents a downstream build using PMC since the fetcher treats ./out/RPMS as a repo the same way as PMC (just higher priority).

Test 2

Now adding to ./SPECS-TEST:

• cloud-init.spec: new epoch version 1:23.3
• cloud-init-old.spec: old version 23.4.1

sudo make clean
sudo make build-packages SPECS_DIR=../SPECS-TEST PACKAGE_REBUILD_LIST=a

The graph selects 1:23.3 as the best version to resolve BuildRequires: cloud-init.

Test3

However, modifying a.spec to have BuildRequires: cloud-init >= 23.4 results in the same selection of 23.3.1. I believe this is expected behavior here. On the bright side, no packages in our repo actually have a (Build)Requires: cloud-init. Also validated with depsearch:

__'.' from node 'Meta(21317)'
   |__'cloud-init-23.4.1-3.cm2.noarch.rpm' from node 'cloud-init-23.4.1-3.cm2-RUN<Meta>'
   |   |__'cloud-init-azure-kvp-23.4.1-3.cm2.noarch.rpm' from node 'cloud-init-azure-kvp-23.4.1-3.cm2-RUN<Meta>'
   |   |   |__'<NO_RPM_PATH>' from node 'ALL'
   |   |...
   |__'cloud-init-23.3-1.cm2.noarch.rpm' from node 'cloud-init-1:23.3-1.cm2-RUN<Meta>'
   |   |__'cloud-init-azure-kvp-23.3-1.cm2.noarch.rpm' from node 'cloud-init-azure-kvp-1:23.3-1.cm2-RUN<Meta>'
   |   |   |...
   |   |...
   |...

[cjp256]

Looks good for the various modes of PPS. Results of marketplace vs "mariner-2-proposed" which is the image containing this build:

Or, more succintly:

I don't see any warnings or errors to be concerned with.