Process.spawn with IO.pipe output fails on Java 21 + warbler

[kalenp]

Using Open3.capture3 fails when running with java21 from a warbler jar with

ERROR: org.jruby.embed.EvalFailedException: (EBADF) Bad file descriptor - ls

This works correctly with java 11 and warbler, or with java 21 and no warbler. This is tested on jruby 9.4.12 and warbler 2.0.5.

I simplified the Open3 code down to this repro:

out_r, out_w = IO.pipe

pid = spawn('ls', out: out_w)

wait_thr = Process.detach(pid)
out_w.close
begin
  out_reader = Thread.new { out_r.read }
  output = out_reader.value
  puts output
ensure
  out_r.close
  wait_thr.join
end

When running this, it will fail at spawn. Spawning without an output pipe works fine in the java21+warbler configuration.

[headius]

Probably an issue acquiring the file descriptor from the pipe. I would have expected more error or warning output though. Shouldn't be hard to fix.

[headius]

Ok I understand now that this case is Java 21 running as a warbled jar... so this is definitely an issue getting at those file descriptors.

In order to unwrap file descriptors from some core JDK IO channels, we have to request open access to those classes at a command line. This usually comes in the form of --add-opens flags, which the JRuby launcher appends from bin/.jruby.module_opts. Without these flags we cannot access those JDK classes and without the real native file descriptors, other native calls will not work properly (raising EBADF here for whatever bogus value we fell back on).

The complication here is that warbler has never been updated to support modules. That feature would provide a way to configure a module-info.class to include in the warbled jar file. That would allow providing a proper module name for the jar, and might also be able to open those JDK packages without additional flags. It needs to happen at some point, along with the remaining full modularization of JRuby itself.

In the absence of that feature, the way to enable this access is to duplicate the way JRuby launches:

• Modify the warbled jar to have an Automatic-Module-Name (optional, allows more narrow opens below).
• Run with the jar on the module-path rather than as an executable jar or on classpath.
• Provide the --add-opens flags adjusted to the module name you chose

The least steps would be:

• Provide the --add-opens flags with the "ALL-UNNAMED" module as the target.

This would open up the JDK packages to the warbled executable jar, but also to everything else not in a module (which is the default behavior that existed prior to Java 17).

[headius]

I've transferred this issue to the warbler repo because it's really a problem with how warbler deploys apps. JRuby itself requires some open modules (fewer as time goes on) and we need to investigate how to make warbler open those modules up on deployment. At the very least, warbler needs to deploy the bundled JRuby as a module, so that --add-opens flags can be used to enable the necessary functionality.