renovate
renovate copied to clipboard
renovatebot
feat(workers/repository): Update package files in the same order they're extracted
Changes
For some managers that can have dependencies between input files (pip-compile, for example) it's important to update each input file in the correct order. This changes getUpdatedPackageFiles to sort its lists of package files into the same order that they were returned by the manager's extract function.
Since package files can be referenced by multiple managers and since each manager could return the package files in a different order I also inverted the update logic from
for each package file
for each manager
manager.update(package file)
to
for each manager with updated package files
sortPackageFiles
for each package file
manager.update(package file)
This lets us sort the list of package files only once per manager.
Context
- #28518
Documentation (please check one with an [x])
- [ ] I have updated the documentation, or
- [x] No documentation update is required
How I've tested my work (please select one)
I have verified these changes via:
- [ ] Code inspection only, or
- [ ] Newly added/modified unit tests, or
- [ ] No unit tests but ran on a real repository, or
- [x] Both unit tests + ran on a real repository
@rarkins @viceice Is there anything I can do to help move this PR along? I understand that this is core functionality and so you want to be cautious with this change; I just wanted to see if there was anything I could do to help you review these changes.
@secustor @zharinov can you help to review and test this core change carefully. so we don't introduce any regression. maybe you've ideas to make the changes more simple. eg by introducing an order index on extract and sort by it after, as we discussed in the dependent discussion. ( I think it was on slack channel)
@viceice Note this is the simplified version. No extra order index was needed because the repo worker still has access to the list of package files as returned by each manager (they're all in the BrachConfig) and can use it to re-sort the final list of modified package files based on its index in that original list.
The extra complexity comes from the fact that multiple managers can update the same package file (in theory, at least), so the list has to be re-sorted before it's passed to each manager's updateArtifacts.
Just quick bump on this since it's been a month since the last activity. Would it help if I split this PR in two and separated the refactor to invert the update logic from the new feature of sorting the package files?
The logic which is getting changed by this PR is very central/important to renovate. Any mistake we miss would affect every user. Right now these changes are hard to review, so any split of the PR to make it simpler would help
Just quick bump on this since it's been a month since the last activity. Would it help if I split this PR in two and separated the refactor to invert the update logic from the new feature of sorting the package files?
Yes, please split out the refactor
Is there any update on this matter? This creates a lot of issues in big repositories where requirements files are dependent from the output of the other. Sadly when renovate is updating them randomly it makes it unusable as every PRs is failing because of RequirementsConflicted error.
@pawel-kankowski-airspace-intelligence I've been busy recently and haven't had a chance to split this PR up. I'll try to get to it this week.
I created #30725 with just the refactor from this PR. I'll update this one with the package sorting once that one is merge.
@viceice @rarkins can you share some time estimate when do you think this can be reviewed?
:tada: This issue has been resolved in version 38.66.0 :tada:
The release is available on:
Your semantic-release bot :package::rocket:
