Allow selective version resolutions for transitive dependencies

[hlascelles]

The problem

Sometimes we are aware that a transitive dependency has a CVE. A bundle update somegem will update that gem in the lockfile temporarily, but it may slip back down later. We want to specify a version for it to ensure it stays above a minimum, but without implying that it is directly depended on by the root project.

This would be the equivalent of the resolutions section in yarn. To the best of my searching ability, I don't think bundler has this functionality.

Proposal

Introduce a resolutions section (like a group) to specify these needed resolutions which are not direct dependencies.

eg:

# Proposed syntax:
resolutions do
  gem "thingparser", "> 0.1.0" # Has CVE!
end

Steps to reproduce the problem

Consider:

### Gemfile
source "https://rubygems.org"

gem "somegem", "~> 1.0.0"  # Has a transitive dependency on 'thingparser'

### Lockfile
GEM
  remote: https://rubygems.org/
  specs:
    somegem (1.0.0)
      thingparser (~> 0.1.0)
    thingparser (0.1.0)

DEPENDENCIES
  somegem (~> 1.0.0)

Now, we learn that thingparser has a CVE, and 0.1.1 is required. I would like to do:

### Gemfile
source "https://rubygems.org"

gem "somegem", "~> 1.0.0"

# Proposed syntax:
resolutions do
  gem "thingparser", "> 0.1.0"
end

### Lockfile
GEM
  remote: https://rubygems.org/
  specs:
    somegem (1.0.0)
      thingparser (~> 0.1.0)
    thingparser (0.1.1)

DEPENDENCIES
  somegem (~> 1.0.0)

Note, the thingparser gem is now specified above 0.1.0, but my intent is clear in the Gemfile that I don't directly depend on it.

We would find this valuable in bundler.

[hsbt]

Does https://github.com/tarnowsc/bundler-override help that usecase?

[hlascelles]

Aha, it's halfway there. It allows direct overriding of a gem, which means that gem appears in the lock file regardless as to whether it is needed later (as I understand it).

However, what if:

a. You have many dependencies that all have a transitive dependency on thingparser. You'd want them all to be raised. Should you create an entry for each one? b. What if a future gem you add has a transitive dependency on thingparser. Will you remember to add an override? c. What if your concrete dependency gem updates and no longer has that transitive dependency. It will remain in the lock file as declared for your concrete dependency gem regardless. This useless addition may hold back your updates later.

So, definitely same ballpark, but different usecase...

Thank you for the pointer, I will maybe make a bundler plugin that demonstrates the solution I am looking for!

[deivid-rodriguez]

Hi! I think the rationale of this makes sense. Indeed, Bundler may end up downgrading an indirect dependency if necessary when requested to upgrade a top level gem. And indeed that's probably no good. Even if you may not be directly affected by the CVE, just the "noise" from security alerts and the difficulty to upgrade the dependency again (since it will require to downgrade back the top level dependency, which most tools will try not to do) make this inconvenient.

Do you have a realworld example of this happening? Do you forsee any usage of this, other than preventing downgrading dependencies to vulnerable versions.

I ask because I wonder if a new Gemfile API is really necessary/useful or if maybe we should consider promoting functionality similar to https://github.com/rubysec/bundler-audit to Bundler, so that Bundler is aware of CVE's and can actively avoid vulnerable versions.

In any case, I think any approach implemented with a Bundler plugin would be a great start!

[hlascelles]

OK, sounds good. I'll give a plugin a whirl as a proof of concept.

If it integrated with bundler-audit that would be interesting. i'll if that can be a phase 2. Thanks!

[deivid-rodriguez]

I still wonder about a realworld example of this happening and if you forsee any usage of this, other than preventing downgrading dependencies to vulnerable versions.

Thank you!

[hlascelles]

I've put together a plugin as an example.

I've called it bundler-resolutions: https://github.com/hlascelles/bundler-resolutions

I couldn't find a clean way of adding it as a standard bundler plugin, so it does some module prepending. I've only tried it in the most vanilla of Gemfiles, but the outcome is what I was trying to achieve.

Usage

Add bundler-resolutions to your Gemfile, and add a resolutions group to specify the gems you want to specify versions requirements for.

The resulting Gemfile.lock in this example will have nokogiri locked to 1.16.5 or above, but it will not appear in the DEPENDENCIES, as we are saying it is not a concrete dependency of the project.

plugin 'bundler-resolutions'

gem "rails"

group :resolutions do
  gem "nokogiri", ">= 1.16.5" # CVE-2024-34459
end

The Gemfile.lock from this example will not have nokogiri at all, as it is neither explicitly declared, nor brought in as a transitive dependency.

plugin 'bundler-resolutions'

group :resolutions do
  gem "nokogiri", ">= 1.16.5" # CVE-2024-34459
end

Let me know what you think.

[deivid-rodriguez]

Thank you! Nice that you managed to get something working! It seems very similar to npm resolutions indeed. What I don't like for the design is using a specially named group that works differently from other groups, did you consider introducing a new DSL?

Also, I would really like some answers for the questions I asked above, because as of now, I'm still unsure if a new feature is better than working towards builtin support for security advisories in Bundler, so that Bundler never downgrades to a vulnerable version of a dependency by default. If the only use case is the one you mentioned in the description, I think that would be a better goal, since Bundler would be more secure, by default, for everyone, no Gemfile changes needed.

[hlascelles]

Great, glad it makes sense...

For the DSL, yes, I did look at trying to add a resolutions block, but I couldn't get my plugin to load early enough in the bundler lifecycle to allow me to alter the DSL. Is the only way to add a (somewhat ugly) explicit load like bundler-override does? I see a proposal to add a plugin lifecycle call at the right point is here. I am cautious though, I like the fact that all Gemfiles look the same with the same DSL.

On the second point, yes, I completely agree with you, building CVE/bundler-audit awareness into Bundler itself is much preferred. It is the main reason I wrote this as a first step, and secure-by-default-for-all is better than a plugin.

A few more thoughts on why we need the plugin:

1. We have a monorepo with a dozen services and a hundred internal gems. We have found upgrading any one third party gem for all of them in lockstep pays huge dividends in building and supporting deploys (with less effort than people might think). One big reason for this plugin is to make use of our blessed versions as a Gemfile fragment, which we could include into every Gemfile in a resolutions block.
2. We have seen over the years a scenario where a junior developer has got themselves in a mess with a lock file, copied in an older version and "fixed it" with bundle install. This can leave a dependency in a regressed/different state to other services, which complicates deploys.

Note that these two scenarios include much more than CVEs, so even a Bundler/bundler-audit native system wouldn't help there. I've more examples in the repo covering areas like legal and OS architecture.

I'll add some more features for our use cases, see if it actually works out. The plugin repo is already marked as experimental. I'll leave it that way and link to this conversation if others want to chime in.

As far as the CVE side goes, I could have a go at a PR to Bundler for you, but I'm sure it needs a lot of thought (which I'm sure you've already done) and good Bundler codebase knowledge? Thank you again for Bundler!

[caius]

Somewhat of a :car::dash: thought, but could we add an option to the gem method where we set dependency: false to exclude the gem from the DEPENDENCIES in the lockfile? Thinking along the lines of require: false being in there already. Alternatively something like pin_gem "foo", ">= x.x.x" instead of gem?

I agree this would be super useful, up till now apps I've worked on have a special block of gem lines with a comment above them stating they are pinned to a specific equal-or-greater-than for security reasons but that the gems aren't part of our app dependencies. Having a way to specify this in an explicit/lightweight fashion would be neat.

The other scenario where the above isn't so great is when the indirect dependency is removed from the library, but because we have gem :… in the Gemfile we continue installing/requiring that gem until we notice nothing depends on it. Could we emit a warning if that case was detected when there's a DSL/group for denoting these pinned indirect dependencies?

[nhumble-sa]

This would be very helpful in addressing vulnerabilities in transitive dependencies when upgrading the direct dependency is not an option at the time. There are two main reasons for this:

1. The direct dependency hasn't updated yet.
2. The direct dependency has updated on a newer version and upgrading would create a lot more work (e.g. rails and rack, there is a current vulnerability in rack that is resolved in the latest version of rails, but that can be a major version upgrade for existing applications).

[hlascelles]

I've done an update to bundler-resolutions. It is also now a normal gem, not a bundler plugin. Also I have removed some of the monkey-patching of bundler and extracted out the versioning config into a yaml file.

The reason for this was that I was finding myself working hard against bundler to make the Gemfile DSL "consider the versions of the resolution gems, but not depend on them".

I agree it would be nice to have a native DSL to express this intent in the Gemfile itself at some point. Great to hear some other ideas here about how the native syntax cold be represented!

Right now, though, you can use the new version of bundler-resolutions, v0.2.0, to perform "version constraint without declaring explicit dependencies".

For example, you can use this setup:

# .bundler-resolutions.yml

gems:
  nokogiri: ">= 1.16.5" # CVE-2024-34459
  sprockets: 3.7.2

# Gemfile

gem "bundler-resolutions"
gem "rails"

With this arrangement we have specified minimum nokogiri and sprockets versions, then declared rails as normal. In the future we will know that other co-workers cannot unintentionally downgrade nokogiri, and that if/when Rails sprockets is removed from rails it will disappear from our lockfile too, and we we will not have a lingering dependency that may hold upgrades back / cause issues.

I will next look at seeing if it is practical to add support for bundler-leak, bundler-audit and others.