Support default extension gems

[eregon]

This an issue to track all default gems with an extension, which might need specific support for TruffleRuby. For some, we might just reuse the C extension and add TruffleRuby in the CI of those gems.

These default extension gems are problematic when they included in a Gemfile.lock, https://bugs.ruby-lang.org/issues/18567#note-30 has more details. Related JRuby issue: https://github.com/jruby/jruby/issues/6682

For some extensions gems, we have a different implementation, typically pure-Ruby. For some extensions gems we use them as-is (just need to add truffleruby in CI).

• [x] bigdecimal: https://github.com/ruby/bigdecimal/pull/247
• [x] cgi: should add to CI and look into failures. JRuby support PR for cgi/escape https://github.com/ruby/cgi/pull/44
• [ ] date
• [x] debug: actually a bundled gem https://github.com/ruby/debug/pull/1039
• [ ] digest: https://github.com/oracle/truffleruby/issues/1390 We probably want to use the C extension to support ruby/digest.h
• [x] etc: https://github.com/ruby/etc/pull/27
• [ ] fcntl
• [ ] fiddle
• [x] io-console: https://github.com/ruby/io-console/pull/23
• [x] io-nonblock: https://github.com/ruby/io-nonblock/pull/12
• [ ] io-wait
• [x] json: https://github.com/flori/json/pull/402
• [ ] openssl
• [ ] pathname
• [x] pp: https://github.com/ruby/pp/pull/20
• [ ] psych
• [ ] racc
• [ ] readline-ext
• [x] stringio: we have a pure-Ruby implementation, similar to strscan https://github.com/ruby/stringio/pull/71 https://github.com/ruby/stringio/pull/74
• [x] strscan: we have a pure-Ruby implementation https://github.com/oracle/truffleruby/issues/2420 https://github.com/ruby/strscan/pull/35
• [ ] syslog
• [x] zlib https://github.com/ruby/zlib/pull/62

Pure-Ruby gems (good to add truffleruby in CI):

• [x] rdoc: https://github.com/ruby/rdoc/pull/1032
• [x] timeout: https://github.com/ruby/timeout/pull/15 the original pure-Ruby impl is too slow with 1 new Thread per call

[chrisseaton]

Do JRuby have an approach to this problem? Do they all include Java code?

[eregon]

Yes basically JRuby's approach so far is to mostly upstream their JRuby-specific extension Java code into those default gems' repositories (and publish a java platform gem). You can check the linked PRs in the linked JRuby issue for more details.

[eregon]

Here is a list of default gems with extensions in 3.1:

$ grep -a -R extensions lib/gems/specifications/default  | sort
lib/gems/specifications/default/bigdecimal-3.1.1.gemspec:  s.extensions = ["ext/bigdecimal/extconf.rb".freeze]
lib/gems/specifications/default/cgi-0.3.5.gemspec:  s.extensions = ["ext/cgi/escape/extconf.rb".freeze]
lib/gems/specifications/default/date-3.2.2.gemspec:  s.extensions = ["ext/date/extconf.rb".freeze]
lib/gems/specifications/default/digest-3.1.0.gemspec:  s.extensions = ["ext/digest/bubblebabble/extconf.rb".freeze, "ext/digest/extconf.rb".freeze, "ext/digest/md5/extconf.rb".freeze, "ext/digest/rmd160/extconf.rb".freeze, "ext/digest/sha1/extconf.rb".freeze, "ext/digest/sha2/extconf.rb".freeze]
lib/gems/specifications/default/etc-1.3.0.gemspec:  s.extensions = ["ext/etc/extconf.rb".freeze]
lib/gems/specifications/default/fcntl-1.0.1.gemspec:  s.extensions = ["ext/fcntl/extconf.rb".freeze]
lib/gems/specifications/default/fiddle-1.1.0.gemspec:  s.extensions = ["ext/fiddle/extconf.rb".freeze]
lib/gems/specifications/default/io-console-0.5.11.gemspec:  s.extensions = ["ext/io/console/extconf.rb".freeze]
lib/gems/specifications/default/io-nonblock-0.1.0.gemspec:  s.extensions = ["ext/io/nonblock/extconf.rb".freeze]
lib/gems/specifications/default/io-wait-0.2.1.gemspec:  s.extensions = ["ext/io/wait/extconf.rb".freeze]
lib/gems/specifications/default/json-2.6.1.gemspec:  s.extensions = ["ext/json/ext/generator/extconf.rb".freeze, "ext/json/ext/parser/extconf.rb".freeze, "ext/json/extconf.rb".freeze]
lib/gems/specifications/default/openssl-3.0.1.gemspec:  s.extensions = ["ext/openssl/extconf.rb".freeze]
lib/gems/specifications/default/pathname-0.2.0.gemspec:  s.extensions = ["ext/pathname/extconf.rb".freeze]
lib/gems/specifications/default/psych-4.0.4.gemspec:  s.extensions = ["ext/psych/extconf.rb".freeze]
lib/gems/specifications/default/racc-1.6.0.gemspec:  s.extensions = ["ext/racc/cparse/extconf.rb".freeze]
lib/gems/specifications/default/readline-ext-0.1.4.gemspec:  s.extensions = ["ext/readline/extconf.rb".freeze]
lib/gems/specifications/default/stringio-3.0.1.gemspec:  s.extensions = ["ext/stringio/extconf.rb".freeze]
lib/gems/specifications/default/strscan-3.0.1.gemspec:  s.extensions = ["ext/strscan/extconf.rb".freeze]
lib/gems/specifications/default/syslog-0.1.0.gemspec:  s.extensions = ["ext/syslog/extconf.rb".freeze]
lib/gems/specifications/default/zlib-2.1.1.gemspec:  s.extensions = ["ext/zlib/extconf.rb".freeze]

And for the raw list:

grep -a -R extensions lib/gems/specifications/default  | sort | ruby -e 'puts STDIN.readlines.map { |line| line[/\/([a-z-]+?)-\d/,1] }'

We should prioritize adding truffleruby in the CI of those to make sure they work and keep working on truffleruby.