Fixing cascading rebuilds. (CP: #8777)
Merge Checklist
All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)
- [X] The toolchain has been rebuilt successfully (or no changes were made to it)
- [X] The toolchain/worker package manifests are up-to-date
- [X] Any updated packages successfully build (or no packages were changed)
- [X] Packages depending on static components modified in this PR (Golang,
*-staticsubpackages, etc.) have had theirReleasetag incremented. - [X] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
- [X] All package sources are available
- [X] cgmanifest files are up-to-date and sorted (
./cgmanifest.json,./toolkit/scripts/toolchain/cgmanifest.json,.github/workflows/cgmanifest.json) - [X] LICENSE-MAP files are up-to-date (
./SPECS/LICENSES-AND-NOTICES/data/licenses.json,./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md,./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON) - [X] All source files have up-to-date hashes in the
*.signatures.jsonfiles - [X]
sudo make go-tidy-allandsudo make go-test-coveragepass - [X] Documentation has been updated to match any changes to the build system
- [x] Ready to merge
Summary
Cherry-pick of #8777.
This fixes and occasional issue, where packages depending on an updated version of another package would not get triggered to re-build despite cascading rebuilds being set to 1.
The issue was caused by the order, in which we verify if a rebuild is needed and propagate the "freshness" of a node. Scenario where the issue reproed:
- Packages:
A,B, andX. - Both
AandBwere modified in the PR. Xhas a build-time dependency onB.Bhas a build-time dependency onA.- Cascading rebuilds are set to 1 meaning "rebuild only the first level of packages depending on the modified packages".
Expected:
In this scenario A and B should build, because they are updated, there's no delta or cache to grab their versions from. Both should also build with the maximum "freshness". Since cascading rebuilds are set to 1, X should also build, because its dependency B was built.
Real:
Once A built, B was unblocked to build. While the scheduler was trying to figure out the "freshness" for B, it first checked if one of its dependencies (in this case A) was built during this run. It was built during this run, so the code decided B needed to be rebuilt but with a "freshness" one lower than A. The logic resetting the "freshness" due to the fact that B was an updated package was then skipped. Once B built, X didn't rebuild, because its dependency B built with "freshness" 0, meaning that was the last package the tooling would try to rebuild in the dependency chain.
Fix:
We now first check, if B needs to build because it was updated and reset its "freshness", if that's the case.
Does this affect the toolchain?
No.
Test Methodology
- PR check with changes from #8673. Pre-fix build, post-fix build.
- Local tests.