dependabot-core
dependabot-core copied to clipboard
dependabot
Support grouping for "must-be-updated-together" dependencies
To be clear, this feature request is different from #1190. It is not about convenience (creating one PR instead of creating multiple PRs to keep the history clean), but rather about something that must-have when you're working with scoped packages or packages that are released together.
The issue:
There are some packages that are released and work together and if you update just one of them the build will essentially fail.
Examples for such packages are @angular/core & @angular/compiler, @angular/cli & @angular-devkit/build-angular, react and react-dom and so on.
The request:
Allow (either implicitly like in greenkeeper) or explicitly (like in renovate) updating these group dependencies in a single PR.
The lack of this functionality is a blocker for anyone who uses multiple packages that are tightly coupled and have the same release schedule (and even a few packages I specified above are, well, quite popular).
All the beautiful work Dependabot does is worth nothing if one still have to spend time on manually updating group dependencies.
@just-jeb we have the ability to group tightly coupled dependencies but have only done so for vue and vue-template-compiler (because it breaks the build if using different versions).
From my understanding react and react-dom minor/patch versions can be out of sync without breaking. Whenever there's a major version bump we'll update them together (due to the peer dependency requirement).
Trying to figure out if we should be grouping angular and seeing that @angular/core, @angular/compiler, @angular/cli have a slightly different versions scheme and release schedule to @angular-devkit/build-angular and @angular/pwa.
You know if these should be grouped differently? Is there anything in angular that breaks if
for example, core and compiler versions don't match up?
Looking at the greenkeeper list it looks like we could maybe just use their definitions list to group these dependencies.
Using greenkeeper definitions is a good start, although making it configurable is a preferable option.
Regarding Angular I know for sure that @angular/cli and @angular-devkit are tightly coupled and can break even with minor version changes. I'd say for Angular everything (the whole @angular scope and @angular-devkit scope) should be grouped together as they are always released together.
There is an open PR (for almost a year now) in Greenkeeper monorepo-definitions repository where you can get more details.
@just-jeb yeah definitely keen to support custom grouping in future but pretty stacked atm integrating with Github so won't get to anytime soon unfortunately. Will see what makes sense on grouping cli/devkit. Keen to keen the grouping to stuff that's breaking atm until we have a way of configuring it.
Just wanted to bump this and report that I've encountered the same issue with graphics and windowing crates in the Rust ecosystem. gfx, glutin, and gfx_window_glutin often require lockstep upgrades and dependabot has opened separate PRs against my repositories to upgrade each of these independently. Those PRs fail CI with no (obvious) way to merge/combine them.
I would love this. For example I would like all NPM packages that start with @babel/ to be updated together and all packages that start with @storybook/ to be updated together.
My npm outdated often looks like this
which would be nice if it was just 2 PRs (although to be fair it's happened before that some specific babel plugin dependency breaks my build but the others don't but I can debug this manually when it happens)
I would like to point out this is also the case for .NET core sdks, there was a recent bump from 2.2 to 3 and we had about 30 prs created that I had to manually consolidate into one branch to get it to build
Worth mentioning that this is also relevant to RubyGems with Sorbet, which should update sorbet, sorbet-static, and sorbet-runtime all at once.
This seems to be related to https://github.com/dependabot/dependabot-core/issues/317
Isn't there a solution for this yet?
Dependant bot spams my email every day to update every single package individually! I prefer to bump all of them at the same time in a single pull request if my tests pass!
Unfortunately, the dependant bot follows a rigid architecture of Ruby classes which makes it hard for adding new features. These features could be a simple shell command rather than a Ruby class. I can do this very easily in using npm-check-updates and GitHub actions.
name: Bump
on:
schedule:
- cron: "0 0 * * *"
jobs:
bump:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2-beta
with:
node-version: '14'
- run: |
npm install -g npm-check-updates
ncu "/^@babel\/.*$/" # or `ncu -u`
npm install
- uses: peter-evans/create-pull-request@v2
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: Update dependencies
title: "[AUTO] Update dependencies"
labels: Dependencies
branch: "Bump"
I haven't been able to find whether this is documented anywhere, but I discovered a workaround (at least for Gradle dependencies): https://github.com/dependabot/dependabot-core/issues/1618#issuecomment-665863295
@ashughes Thanks for the answer. That seems to be specific to Gradle. There is no such thing in other languages. That is why a shell coding interface is necessary for the Dependant bot to allow others to execute custom dependency management tasks.
@aminya To be clear, I wasn't suggesting that this issue is solved. Just that for Gradle there appears to be a workaround (and I'm not sure if there are similar workarounds for other build systems). Still would like to see official support for this in Dependabot!
The answer is simple: talk with the guys in charge of package.json syntax and add a variable interpolation as is in maven or gradle. I miss that feature on node that is on java for decades. Upgrading angular dependencies is a hell without having such a feature. I think is not a dependabot problem but a package.json problem.
@mercuriete That is already solved by ncu. The problem is that there is no shell interface for dependant bot.
See my comment: https://github.com/dependabot/dependabot-core/issues/1296#issuecomment-657438154
Just my 2 cents on this. I'm really struggling with this missing capability on my repositories :(
I think I can workaround some like it was said above with a variable for some external libraries where the numbers match, but that is not always the case. The main issue is when there are many upstream repositories that contribute to mine, and the version numbers are all independent and cause most builds to fail with "downgrade detected" errors.
Note: in my scenario it is for C#/nuget projects
Just another use case: My project uses an electron, and during build requires electron-releases which provide additional information about the current version of the electron.
Thus electron-releases should ALWAYS be updated before the electron is updated, or updated simultaneously. Otherwise, this will cause the build to crash.
Because the bot creates separate update PR, the team must keep this feature in mind so as not to accidentally merge the electron update PR before merging the electron-releases PR.
I ran into this issue in my repo when dependabot updated @fortawesome/free-solid-svg-icons (https://github.com/sparkletown/sparkle/pull/872), but not the other related deps:
-
@fortawesome/free-regular-svg-icons -
@fortawesome/fontawesome-svg-core -
@fortawesome/react-fontawesome
I ended up implementing (https://github.com/sparkletown/sparkle/pull/973) a (rather unconventional) workaround to add a test that fails when one of the 'related deps' is updated without the others:
- https://github.com/sparkletown/sparkle/blob/307eb1cc672cca4a806a327b3e313f9af3ebfed6/package-lock.test.ts#L16-L40
I have done similar in the past to check the various React versions and ensure they all match as well.
Not a perfect solution at all, but at least it makes it obvious and we can easily manually step in and fix it when it happens.
I had a PR like that a year ago for jest so I this is something new. I've only ever had one like this in the hundred or so repos that I have using Dependabot though.
https://github.com/xt0rted/slash-command-action/pull/100
This is still on the list of things we'd love to fix but haven't had the bandwidth yet. We've been spent a lot of time keeping up with growth and lately focusing on upgrading package managers.
The above grouped update example happens when there's a peer dependency requirement between several dependencies forcing them to be updated together. The current logic isn't perfect and dependabot often bails trying to resolve these conflicts and opts to do nothing instead :/
Another use case (recognizing that you have it on the roadmap, and putting it here for search engines mostly): @types for specific JS packages. It's pretty often going to be the case—especially across any major version bump—that you will need to upgrade @types/foo-package and foo-package together.
Today we got a PR from Dependabot that updates vue and vue-template-compiler together (npm), so it seems like Dependabot somehow supports this already in some cases?
Also an issue with .net library sets like EF and 3rd parties that depend on these libraries.
Can confirm that this is an issue on .NET. Various Microsoft.EntityFrameworkCore.* packages (Microsoft.EntityFrameworkCore.Abstractions, Microsoft.EntityFrameworkCore.Analyzers, Microsoft.EntityFrameworkCore.Design, etc.) can't be updated individually because mixing of different versions of these packages is not supported. Dependabot created 7 PRs for each of those packages and most of those PRs failed to build. I had to update everything manually and then dependabot closed its PRs.
A use case for golang is k8s dependencies. All packages from "k8s.io" need to be updated at the same time.
Example go.mod snippet
k8s.io/api v0.20.4
k8s.io/apimachinery v0.20.4
k8s.io/client-go v0.20.4
k8s.io/kubectl v0.20.4
k8s.io/kubernetes v1.20.4
Would be nice for @aws-cdk packages as well (node and others). They are all kept to the same version and actually must be deployed with the same version.
Any timeline for this beloved feature?
Got a grouped update today. I don't know how they are managed.
https://github.com/wallabag/wallabag/pull/5537
@babel packages aren't grouped by the way.
I'm wondering how some packages are grouped and some aren't.
Solving this is a more general way is a bit beyond dependabot's scope. However, you should be able to solve this with a custom github workflow. You would create an action that runs whenever a new PR is created by dependabot and it uses github-script to locate instances of PRs for related dependencies across your repo and then merges the changes from the new PR into the existing one and closes this new PR with a comment referring to the other PR.
Related to #3856 and #1190 (and possibly #2652)
Has this been implemented now? I just had this PR:
I am pretty sure this happens because ajv is defined as a peer dependency for ajv-keywods. https://github.com/ajv-validator/ajv-keywords/blob/master/package.json#L40
