ruby
ruby copied to clipboard
ruby
Remove no longer used RUBY_ABI_VERSION
Sharing my thoughts about this here too in addition to Slack:
Users of the ABI version are e.g. gems testing ruby-head using ruby/setup-ruby. If they have many CI runs and lots of gems to install it's a significant overhead on their CI and on RubyGems.org to always reinstall all gems vs using the cache.
If there is no ABI version we can only use the git commit, which is quite wasteful, do we want to waste resources like this for little effort to maintain the ABI version? TruffleRuby maintains a correct ABI version for years now, it's very little overhead, especially if it doesn't change often.
It needs an approach where it's explicitly tracked in CI though, whenever a include/ file changes, either the ABI version is bumped or another file is bumped to acknowledge it's ABI-compatible.
e.g. in TruffleRuby, if a header changes, then one must either bump https://github.com/oracle/truffleruby/blob/master/lib/cext/ABI_version.txt or https://github.com/oracle/truffleruby/blob/master/lib/cext/ABI_check.txt. The check is at https://github.com/oracle/truffleruby/blob/5dbdd3187e8a03435abe4560f9472a7b835f3073/tool/jt.rb#L3121
Other users of the ABI versions are Rubyists using ruby-build 3.3-dev/ruby-dev etc. Those risk segfaults/memory corruption if gems are reused due to CRuby not setting a correct ABI version (i.e. same "version" for ABI-incompatible changes). It would be a clear early error instead with the ABI version check.
There is more reasoning at https://bugs.ruby-lang.org/issues/18249. Removing the ABI version breaks all of these. So if this is removed, I think the least is to create a ticket at https://bugs.ruby-lang.org/ and explain how all the issues of https://bugs.ruby-lang.org/issues/18249 will be solved then.
I heard Shopify once used RUBY_ABI_VERSION, but after some practice they give up to use it because what people can do is only clean and build when they hit ABI incompatibility. Do you use RUBY_ABI_VERSION and how it makes your life easier?
because what people can do is only clean and build when they hit ABI incompatibility.
They'll probably hit a segfault in such cases, with the ABI version it's a nice and clean error instead. If it's not someone working on CRuby I think they would be most confused to get a segfault, and even if they are it might still be confusing and waste a fair amount of time until they figure out what it is. So it seems a loss-loss situation to remove it. The added complexity is negligible.
Also note that Bundler uses the ABI version automatically e.g. with path set (often set to vendor/bundle).
So as long as the version is correctly maintained, you cannot hit a ABI incompatibility segfault when using Bundler with path, which is a really nice property, which is lost without the ABI version.
It's really easy to maintain the ABI version correctly, we have been doing it for years in TruffleRuby. It can easily be checked in CI. I don't understand why some CRuby devs want to remove this so much, I think they largely overestimate the costs. It would also be possible to automatically compute an ABI version based on all include/ files, which once integrated really has 0 cost. So why not do that and prefer to let Rubyists using ruby-head to get confusing segfaults instead? (which might also mean more segfault reports at https://bugs.ruby-lang.org/).
Do you use RUBY_ABI_VERSION and how it makes your life easier?
Probably no one uses the RUBY_ABI_VERSION macro, but many places use the ABI version accessible via RbConfig::CONFIG['ruby_version']. Merging this PR means that ABI version becomes incorrect for ruby-head, while being the correct thing to use for releases. So it breaks, for non-releases, every usage of RbConfig::CONFIG['ruby_version'], which is many: https://github.com/search?q=RbConfig%3A%3ACONFIG%5B%27ruby_version%27%5D+OR+RbConfig%3A%3ACONFIG%5B%22ruby_version%22%5D&type=code
I don't understand why some CRuby devs want to remove this so much, I think they largely overestimate the costs.
I want a release version is as similar as a development branch because such difference cause troubles multiple times in release process. Especially ABI version, it caused build failure on 2022-12-25.
It would also be possible to automatically compute an ABI version based on all include/ files, which once integrated really has 0 cost.
It should be implemented who supports ABI version but it is not implemented yet as of today.
which might also mean more segfault reports at https://bugs.ruby-lang.org/
I don't think we received less segfault reports after ABI version is introduced. Moreover the ABI version of head is still 3.3.0+0 even though it has many ABI changes including hiding rb_io_t. I assume no one uses ABI version.
I want a release version is as similar as a development branch because such difference cause troubles multiple times in release process.
I can't find it anymore but IIRC we discussed at some point whether to keep the ABI symbol in releases or not. And IIRC we agreed to keep it but then later things changed. I think that's the solution, just keep it always, then it's consistent. Because indeed there should be the minimum amount of changes between a dev build and a release.
It would still be 3.3.0+N in dev builds and 3.3.0 for the release, does that cause problems?
Especially ABI version, it caused build failure on 2022-12-25.
Which specifically do you mean? I remember issues due to the ABI symbol not present in the release (discussed above, easy to fix by keeping it).
It should be implemented who supports ABI version but it is not implemented yet as of today.
https://github.com/ruby/ruby/pull/5474 originally did this. But someone (I don't recall who) objected and thought it's enough to manually maintain it. Experience has proven that wrong. So we could reapply that approach fairly easily I think.
I assume no one uses ABI version.
It's not true since e.g. ruby/setup-ruby, Bundler (with path) and RubyGems (in the gem_home/extensions dir) all use the ABI version. But I guess people got used to CRuby not maintaining a proper ABI version for ruby-head and didn't take much advantage of it yet (+ as you say it's already not maintained correctly).
So I see 2 big problems if the ABI version is removed:
- ruby/setup-ruby bundler caching for ruby-head will need to use the git commit as the ABI version. This is very wasteful and costs money to rubygems.org. Do you realize merging this PR has that impact?
- Dealing with native extensions + unreleased ruby versions is painful, most notably it takes a long time to recompile them and without an ABI version there is no way to cache it efficiently (i.e. using the commit hash is very inefficient although better than nothing). This is notably the 3rd major problem mentioned in https://github.com/Shopify/ruby/issues/525#issuecomment-1634964830. Both productivity for developers using ruby-head and CI/staging could benefit significantly from this. Benchmarking with ruby-head and many gems is also much more expensive and/or much slower due to that.
Let me cc @k0kubun @casperisfine @byroot and @peterzhu2118 so they are aware of this PR.
Of course we need to fix so the ABI version is tracked properly, e.g., using one of the two solutions described above. I can contribute the one with a CI check + bumping the ABI version file or the ABI check file.
I heard Shopify once used RUBY_ABI_VERSION, but after some practice they give up to use it because what people can do is only clean and build when they hit ABI incompatibility.
We do still "use" it. As in it's part of our gems cache keys and such. But we don't really use it, well because it's a manual bump and as multiple noticed, no-one ever think of bumping it. If it was automatic (or manual but enforced on CI) then it would be very useful for us.
We regularly encounter issues that would be solved if RUBY_ABI_VERSION was reliable.
