Bundler/rubygems generates a lot of exceptions

[eregon]

Extracted from #1398, to keep track more easily of it.

@deepj noticed bundler/rubygems generates a lot of exceptions

See https://gist.github.com/deepj/9a6dccfadd621780242a698274f7396a

This was generated using RUBYOPT=-d ruby bug.rb where bug.rb comes from #1895

MRI produces much less exceptions: https://gist.github.com/deepj/cb29e92d8eba43129ed107c04adf3d83

Originally posted by @deepj in https://github.com/oracle/truffleruby/issues/1398#issuecomment-585533860

[eregon]

lib/mri/bundler/dep_proxy.rb:45:in `method_missing' - undefined method `to_ary' for <Gem::Dependency type=:runtime name="activemodel" requirements="= 6.0.2">:Gem::Dependency

is present on both TruffleRuby and MRI, so it probably comes from Bundler code. It would be useful to investigate what calls to_ary, by using --backtraces-raise.

This part appears only on TruffleRuby:

Exception: `Errno::ENOENT' ~/.rubies/truffleruby-20.1.0-dev/lib/mri/fileutils.rb:1437:in `handle' - No such file or directory
Exception `NoMethodError' at ~/.rubies/truffleruby-20.1.0-dev/lib/mri/rubygems/package/tar_reader.rb:72:in `each' - undefined method `seek' for #<Zlib::GzipReader:0x9e3e0>:Zlib::GzipReader
Exception `Errno::ENOENT' - No such file or directory - ~/.gem/truffleruby/2.6.5/gems/i18n-1.8.2/lib/i18n/tests.rb
Exception: `Errno::ENOENT' ~/.rubies/truffleruby-20.1.0-dev/lib/mri/rubygems/package.rb:460:in `handle' - No such file or directory - ~/.gem/truffleruby/2.6.5/gems/i18n-1.8.2/lib/i18n/tests.rb
Exception `Errno::ENOENT' - No such file or directory - ~/.gem/truffleruby/2.6.5/gems/i18n-1.8.2/lib/i18n/tests.rb
Exception: `Errno::ENOENT' ~/.rubies/truffleruby-20.1.0-dev/lib/mri/fileutils.rb:1307:in `handle' - No such file or directory - ~/.gem/truffleruby/2.6.5/gems/i18n-1.8.2/lib/i18n/tests.rb
Exception `Errno::ENOENT' - No such file or directory - ~/.gem/truffleruby/2.6.5/gems/i18n-1.8.2/lib/i18n/tests.rb
Exception: `Errno::ENOENT' ~/.rubies/truffleruby-20.1.0-dev/lib/mri/fileutils.rb:1307:in `handle' - No such file or directory - ~/.gem/truffleruby/2.6.5/gems/i18n-1.8.2/lib/i18n/tests.rb
Exception `Errno::ENOENT' - No such file or directory

Note that MRI 2.7.0 is used, so there might be a different fileutils.rb version, etc.

@deepj Could you post a log with Ruby 2.6.5 to make sure we're comparing the same standard libraries?

[deepj]

@eregon https://gist.github.com/deepj/6c7b67d2b898003ece448fc21a4444e4

this is more similar to the original gist. But still less exceptions. What it's original for TruffleRuby is such as

Exception `Errno::EINPROGRESS' - Operation now in progress - connect(2)
Exception: `Errno::EINPROGRESS' ~/.rubies/truffleruby-20.1.0-dev/lib/truffle/socket/truffle/error.rb:53:in `handle' - Operation now in progress - connect(2)
Exception `Errno::EINPROGRESS' - Operation now in progress - Operation now in progress - connect(2)
Exception: `IO::EINPROGRESSWaitWritable' ~/.rubies/truffleruby-20.1.0-dev/lib/truffle/socket/truffle/error.rb:85:in `raise_wrapped_error' - unknown error - Operation now in progress - connect(2)
Exception `Errno::EISCONN' - Transport endpoint is already connected - connect(2)
Exception: `Errno::EISCONN' ~/.rubies/truffleruby-20.1.0-dev/lib/truffle/socket/truffle/error.rb:53:in `handle' - Transport endpoint is already connected - connect(2)
Fetching gem metadata from https://rubygems.org/....Exception: `EOFError' ~/.rubies/truffleruby-20.1.0-dev/lib/mri/bundler/vendor/fileutils/lib/fileutils.rb:1291:in `readpartial' - EOFError
Exception `Errno::EINPROGRESS' - Operation now in progress - connect(2)
Exception: `Errno::EINPROGRESS' ~/.rubies/truffleruby-20.1.0-dev/lib/truffle/socket/truffle/error.rb:53:in `handle' - Operation now in progress - connect(2)
Exception `Errno::EINPROGRESS' - Operation now in progress - Operation now in progress - connect(2)
Exception: `IO::EINPROGRESSWaitWritable' ~/.rubies/truffleruby-20.1.0-dev/lib/truffle/socket/truffle/error.rb:85:in `raise_wrapped_error' - unknown error - Operation now in progress - connect(2)
Exception `Errno::EISCONN' - Transport endpoint is already connected - connect(2)
Exception: `Errno::EISCONN' ~/.rubies/truffleruby-20.1.0-dev/lib/truffle/socket/truffle/error.rb:53:in `handle' - Transport endpoint is already connected - connect(2)

[deepj]

One thing, isn't generating exception more expensive in TruffleRuby then in MRI? What I remember JRuby has/had it in this way

[eregon]

I'm glad I asked, it seems these ENOENT happen on MRI 2.6.5 too. Which sounds like FileUtils or zlib is quite a bit better for exceptions in MRI 2.7+.

The EINPROGRESS are not so frequent, about 32 in total. Those are due to our FFI-based Socket implementation. We could probably do it without exceptions.

One thing, isn't generating exception more expensive in TruffleRuby then in MRI?

Probably, yes, since we optimize the stack more (e.g., use the native stack for Ruby values but MRI doesn't) then it's more expensive to de-optimize and to walk it.

OTOH, for very simple cases of the exception being caught not far from where it's raised, if compiled and inlined then exceptions have almost no cost as they get escape analyzed and no stack walking happens. That's still a rare case, because to work the rescue block needs to not access the Exception object (otherwise we need to materialize the backtrace/walk the stack eagerly when entering the rescue).

[eregon]

It seems to be the same exceptions on CRuby and TruffleRuby.