TruffleRuby's support for Childprocess gem

[deepj]

I'm not sure, if this is the right place to put in here. childprocess is used by selenium-webdriver (see https://rubygems.org/gems/childprocess/reverse_dependencies)

selenium-webdriver gem is used by many testing libraries such as capybara. childprocess is not under active maintenance. In fact, a new maintainer is needed (https://github.com/enkessler/childprocess/issues/131).

The issue is, it uses fork under TruffleRuby (which is not supported).

[eregon]

childprocess looks very similar to Process.spawn. I had a quick look at the backends:

• The posix_spawn backend has races for anything using the current working directory (Dir.pwd), so it seems we should avoid it (it's unfortunately not so easy to fix): https://github.com/enkessler/childprocess/blob/c005912c7e13422bc036ec59eaa9dc1570d78d4d/lib/childprocess/unix/posix_spawn_process.rb#L43
• The fork+exec backend uses Ruby-level fork and exec, expecting to run Ruby code in the child process. This is very hard to support on JVM (aka, the JVM currently crashes in this situation).
• The JRuby backend relies on Java interop, which currently does not work on SVM (but that might change).

Probably the best solution here is adding a backend using Process.spawn, which is the official Ruby core API for spawning subprocesses.

[colorfulsing]

hi guys, first of all, I want to thank you all for the amazing work that you guys have done with truffleruby.

Is there any update on this feature?

I have a heavy threaded app that uses selenium web driver within 50 threads that would really benefit by truffleruby. I have been looking forward to implement it on this app for a the last 2 years :)

[eregon]

@colorfulsing Did you see any issue from childprocess, can you post it here or in a gist?

Now we have FFI implemented, so I think the posix_spawn backend should work as good as on MRI. The mentioned race above also exists on MRI, so I guess it's not any worse on TruffleRuby.

[eregon]

Actually the posix_spawn backend is not the default: https://github.com/enkessler/childprocess/blob/c6260739e86c1cabd9baebd76f2058eea2b1ae16/lib/childprocess.rb#L73

$ cruby -rchildprocess -e 'p process = ChildProcess.build("echo", "foo")'                                                  
#<ChildProcess::Unix::ForkExecProcess ...

If we activate it explicitly:

$ CHILDPROCESS_POSIX_SPAWN=1 jt ruby -rchildprocess -e 'p process = ChildProcess.build("echo", "foo"); process.io.inherit!; process.start; process.wait'
<internal:core> core/type.rb:105:in `coerce_to_failed': Coercion error: #<ChildProcess::Unix::PosixSpawnProcess::Argv:0xb48 @ptrs=[#<Truffle::FFI::MemoryPointer address=0x7f95ccd0fe30>, #<Truffle::FFI::MemoryPointer address=0x7f95ccd1a6c0>, #<Truffle::FFI::Pointer address=0x0>]>.to_ptr => Truffle::FFI::Pointer failed (TypeError)
	from <internal:core> core/type.rb:93:in `execute_coerce_to'
	from <internal:core> core/type.rb:85:in `coerce_to'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:203:in `get_pointer_value'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:155:in `convert_ruby_to_native'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:81:in `block in call'
	from <internal:core> core/enumerable.rb:354:in `each_with_index'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:80:in `call'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:120:in `posix_spawnp'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:51:in `block (2 levels) in launch_process'
	from <internal:core> core/dir.rb:301:in `chdir'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:43:in `block in launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `synchronize'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/abstract_process.rb:81:in `start'
	from -e:1:in `<main>'
<internal:core> core/truffle/ffi/pointer_access.rb:684:in `put_array_of_pointer': wrong number of arguments (given 2, expected 1) (ArgumentError)
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:103:in `to_ptr'
	from <internal:core> core/type.rb:91:in `execute_coerce_to'
	from <internal:core> core/type.rb:85:in `coerce_to'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:203:in `get_pointer_value'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:155:in `convert_ruby_to_native'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:81:in `block in call'
	from <internal:core> core/enumerable.rb:354:in `each_with_index'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:80:in `call'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/ffi_backend/function.rb:120:in `posix_spawnp'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:51:in `block (2 levels) in launch_process'
	from <internal:core> core/dir.rb:301:in `chdir'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:43:in `block in launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `synchronize'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/unix/posix_spawn_process.rb:42:in `launch_process'
	from /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/gems/gems/childprocess-4.0.0/lib/childprocess/abstract_process.rb:81:in `start'
	from -e:1:in `<main>'

Which sounds like a FFI issue, we'll try to look at that.

The better solution would be to use Process.spawn though, so I'll file an issue on the gem repo for that (https://github.com/enkessler/childprocess/issues/172).

[colorfulsing]

hi @eregon , thank you very much for the quick response.

I'm currently testing using standalone truffleruby 20.3.0, childprocess 3.0.0 and selenium 3.142.7 within a docker container using image ruby:2.6.6-slim-stretch as base and I have also unset both GEM_HOME and GEM_PATH env vars so thruffleruby works correctly using it's own gem path.

I have also installed these packages on the docker image:

curl
build-essential
libpq-dev
default-libmysqlclient-dev
apt-transport-https
gettext-base
sqlite3
make
clang
llvm
libc++-dev
libc++abi-dev
libssl-dev
libz-dev
libxslt-dev
libxml2-dev
yarn
nodejs
chromium
chromium-driver
xvfb

And also installed pkg-config gem on truffleruby, here is the log:

/root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:176:in `stop_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:123:in `stop'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:115:in `stop'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/platform.rb:150:in `block in exit_hook'
        from <internal:core> core/exception.rb:126:in `backtrace'
        from <internal:core> core/exception.rb:126:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'
/root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:183:in `stop_server'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:118:in `stop'
        from <internal:core> core/truffle/exception_operations.rb:113:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:763:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `new'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:113:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:119:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:763:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `new'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-20.3.0-linux-amd64/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:113:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:119:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:146:in `get_formatted_backtrace'

And this is the log when using truffleruby-head from yesterday:

/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:38: warning: already initialized constant FFI::Platform::OS
/root/truffleruby-head/lib/truffle/ffi/platform.rb:38: warning: previous definition of OS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:59: warning: already initialized constant FFI::Platform::OSVERSION
/root/truffleruby-head/lib/truffle/ffi/platform.rb:59: warning: previous definition of OSVERSION was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:61: warning: already initialized constant FFI::Platform::CPU
/root/truffleruby-head/lib/truffle/ffi/platform.rb:61: warning: previous definition of CPU was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:63: warning: already initialized constant FFI::Platform::ARCH
/root/truffleruby-head/lib/truffle/ffi/platform.rb:63: warning: previous definition of ARCH was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:91: warning: already initialized constant FFI::Platform::IS_GNU
/root/truffleruby-head/lib/truffle/ffi/platform.rb:91: warning: previous definition of IS_GNU was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:92: warning: already initialized constant FFI::Platform::IS_LINUX
/root/truffleruby-head/lib/truffle/ffi/platform.rb:92: warning: previous definition of IS_LINUX was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:93: warning: already initialized constant FFI::Platform::IS_MAC
/root/truffleruby-head/lib/truffle/ffi/platform.rb:93: warning: previous definition of IS_MAC was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:94: warning: already initialized constant FFI::Platform::IS_FREEBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:94: warning: previous definition of IS_FREEBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:95: warning: already initialized constant FFI::Platform::IS_NETBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:95: warning: previous definition of IS_NETBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:96: warning: already initialized constant FFI::Platform::IS_OPENBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:96: warning: previous definition of IS_OPENBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:97: warning: already initialized constant FFI::Platform::IS_DRAGONFLYBSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:97: warning: previous definition of IS_DRAGONFLYBSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:98: warning: already initialized constant FFI::Platform::IS_SOLARIS
/root/truffleruby-head/lib/truffle/ffi/platform.rb:98: warning: previous definition of IS_SOLARIS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:99: warning: already initialized constant FFI::Platform::IS_WINDOWS
/root/truffleruby-head/lib/truffle/ffi/platform.rb:99: warning: previous definition of IS_WINDOWS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:100: warning: already initialized constant FFI::Platform::IS_BSD
/root/truffleruby-head/lib/truffle/ffi/platform.rb:100: warning: previous definition of IS_BSD was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:105: warning: already initialized constant FFI::Platform::NAME
/root/truffleruby-head/lib/truffle/ffi/platform.rb:105: warning: previous definition of NAME was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:106: warning: already initialized constant FFI::Platform::CONF_DIR
/root/truffleruby-head/lib/truffle/ffi/platform.rb:106: warning: previous definition of CONF_DIR was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:110: warning: already initialized constant FFI::Platform::LIBPREFIX
/root/truffleruby-head/lib/truffle/ffi/platform.rb:110: warning: previous definition of LIBPREFIX was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:119: warning: already initialized constant FFI::Platform::LIBSUFFIX
/root/truffleruby-head/lib/truffle/ffi/platform.rb:119: warning: previous definition of LIBSUFFIX was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/platform.rb:131: warning: already initialized constant FFI::Platform::LIBC
/root/truffleruby-head/lib/truffle/ffi/platform.rb:131: warning: previous definition of LIBC was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:32: warning: already initialized constant FFI::USE_THIS_PROCESS_AS_LIBRARY
/root/truffleruby-head/lib/truffle/ffi/library.rb:32: warning: previous definition of USE_THIS_PROCESS_AS_LIBRARY was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:32: warning: already initialized constant FFI::CURRENT_PROCESS
/root/truffleruby-head/lib/truffle/ffi/library.rb:32: warning: previous definition of CURRENT_PROCESS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:79: warning: already initialized constant FFI::Library::CURRENT_PROCESS
/root/truffleruby-head/lib/truffle/ffi/library.rb:79: warning: previous definition of CURRENT_PROCESS was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:80: warning: already initialized constant FFI::Library::LIBC
/root/truffleruby-head/lib/truffle/ffi/library.rb:80: warning: previous definition of LIBC was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/library.rb:182: warning: already initialized constant FFI::Library::FlagsMap
/root/truffleruby-head/lib/truffle/ffi/library.rb:182: warning: previous definition of FlagsMap was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/struct_layout_builder.rb:100: warning: already initialized constant FFI::StructLayoutBuilder::NUMBER_TYPES
/root/truffleruby-head/lib/truffle/ffi/struct_layout_builder.rb:100: warning: previous definition of NUMBER_TYPES was here
/root/truffleruby-head/lib/gems/gems/ffi-1.13.1/lib/ffi/version.rb:2: warning: already initialized constant FFI::VERSION
/root/truffleruby-head/lib/truffle/ffi/version.rb:2: warning: previous definition of VERSION was here
200
/root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:176:in `stop_process'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:123:in `stop'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:115:in `stop'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/platform.rb:150:in `block in exit_hook'
        from <internal:core> core/exception.rb:126:in `backtrace'
        from <internal:core> core/exception.rb:126:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'
/root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:188:in `assert_started': process not started (ChildProcess::Error)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/process.rb:31:in `exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:193:in `process_exited?'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:183:in `stop_server'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:118:in `stop'
        from <internal:core> core/truffle/exception_operations.rb:138:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:758:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:138:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:144:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'
<internal:core> core/kernel.rb:758:in `fork': fork is not available (NotImplementedError)
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/unix/fork_exec_process.rb:20:in `launch_process'
        from /root/truffleruby-head/lib/gems/gems/childprocess-3.0.0/lib/childprocess/abstract_process.rb:81:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:172:in `start_process'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:110:in `block in start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/socket_lock.rb:41:in `locked'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/service.rb:108:in `start'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:303:in `service_url'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome/driver.rb:40:in `initialize'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:46:in `for'
        from /root/truffleruby-head/lib/gems/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver.rb:88:in `for'
        from ./cookie_concept_test.rb:226:in `init_cookies'
        from ./cookie_concept_test.rb:700:in `test'
        from ./cookie_concept_test.rb:749:in `<main>'
        from <internal:core> core/truffle/exception_operations.rb:138:in `append_causes'
        from <internal:core> core/truffle/exception_operations.rb:144:in `append_causes'
        from <internal:core> core/exception.rb:148:in `full_message'
        from <internal:core> core/truffle/exception_operations.rb:171:in `get_formatted_backtrace'

Let me know if I can help you with anything else, and thanks again for the help! :D

[colorfulsing]

It seems that it requires fork to be implemented on core/kernel.rb, not sure how it works to be honest.

[eregon]

Yes, the fork+exec backend of childprocess has no chance to work on TruffleRuby, JRuby and on Windows, fork() is simply not implementable there (well, maybe on TruffleRuby+Native but it would be total overkill for the purpose of childprocess). That's actually something we need to fix in the childprocess gem, it should check Process.respond_to?(:fork) before attempting to use the fork+exec backend, or just always pick the posix-spawn backend on TruffleRuby.

You can set CHILDPROCESS_POSIX_SPAWN=1 to force the posix-spawn backend, but then you'll most likely see the same issue as I posted above (https://github.com/oracle/truffleruby/issues/1525#issuecomment-738825339). That's probably relatively easy to solve on the TruffleRuby side.

[colorfulsing]

I see, that's fine, I'm ok to add new env vars on our servers to avoid fork+exec, so I guess I will need to wait a little longer until the other issue is fixed before using truffleruby.

I'm really eager tho, that means that there is not much left before I'm able to implement it on our app :D

Thanks for the info!

[eregon]

The fix in the TruffleRuby FFI backend was fairly trivial: ff8619d42bd0c39af2c907624560532f86de0045 So now

CHILDPROCESS_POSIX_SPAWN=1 truffleruby -rchildprocess -e 'p process = ChildProcess.build("echo", "foo"); process.io.inherit!; process.start; process.wait'

works.

What's left is auto-detecting this in childprocess and/or https://github.com/enkessler/childprocess/issues/172 so it'd work without any special setting (CHILDPROCESS_POSIX_SPAWN=1 is needed until then).

[eregon]

selenium-webdriver no longer uses childprocess (in upcoming releases) and the test suite passes on truffleruby :tada: https://github.com/SeleniumHQ/selenium/issues/11251#issuecomment-1314051027

[eregon]

This is now properly fixed in childprocess 5 :tada: https://github.com/enkessler/childprocess/pull/175 https://github.com/enkessler/childprocess/releases/tag/v5.0.0