ruby-install
ruby-install copied to clipboard
postmodern
Having trouble with OpenSSL 3 & MacPorts.
I'm not sure how to debug this further, but with OpenSSL 3 installed along side OpenSSL 1.1, I cannot install Ruby 2.7.5 (and probably others).
With only OpenSSL 1.1 installed, the following works:
ruby-install 2.7.5 -- --with-openssl-dir=/opt/local/libexec/openssl11/
However after installing OpenSSL 3, I get the following issue:
ruby-install 2.7.5 -- --with-openssl-dir=/opt/local/libexec/openssl11/
...
*** Following extensions are not compiled:
openssl:
Could not be configured. It will not be installed.
/Users/samuel/src/ruby-2.7.5/ext/openssl/extconf.rb:111: OpenSSL >= 1.0.1, < 3.0.0 or LibreSSL >= 2.5.0 is required
Check ext/openssl/mkmf.log for more details.
...
cat ~/src/ruby-2.7.5/ext/openssl/mkmf.log
...
"gcc -I../../.ext/include/arm64-darwin21 -I../.././include -I../.././ext/openssl -I/opt/local/libexec/openssl3/include -I/opt/local/libexec/openssl11//include -I/opt/local/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -g -O2 -pipe -m64 -c conftest.c"
It looks like it's including both OpenSSL 3 and OpenSSL 1.1 which is probably why the problem happens.
Not sure why this is happening. Maybe there are multiple conflicting configure arguments being provided or autodetected?
Fun coincidence, I got a truffleruby PR related to openssl 3 today, although I guess it's really caused by MacPorts adding support for OpenSSL 3 recently? FWIW this is what TruffleRuby does about it: https://github.com/oracle/truffleruby/pull/2557/files
In any case I think you would get the same without ruby-install, hence it's an issue of CRuby or of not configuring the build properly.
It might work better by setting PKG_CONFIG_PATH:
https://github.com/oracle/truffleruby/blob/6265d32489fcec7cdaf97118322f7b87c90eb2ac/lib/truffle/truffle/openssl-prefix.rb#L48-L50
Odd, ruby-install already has specific logic for MacPorts and will pass in --opt-dir=/opt/local, so I assume --with-openssl-dir=/opt/local/... would be redundant?
https://github.com/postmodern/ruby-install/blob/e5ad58d48fcb2e6e37f8ba8d480fd6e694e46957/share/ruby-install/ruby/functions.sh#L33-L34
This also feels like an upstream CRuby/openssl (gem) bug. Specifying --with-openssl-dir= should override any CRuby/openssl's configure checks for other OpenSSL directories. Ruby 2.7 is still maintained, so it might be worth submitting a bug (if it already hasn't been, I searched but couldn't find a similar bug) and trying to get a patch backported to ruby 2.7.
It certainly could be a bug with 2.7 and it seems fixed I tested 3.0.3 and 3.1.0
Odd,
ruby-installalready has specific logic for MacPorts and will pass in--opt-dir=/opt/local, so I assume--with-openssl-dir=/opt/local/...would be redundant?
I think the default one gives OpenSSL 3.0, which isn’t supported by Ruby 2.7 (because Ruby 2.7 was released before OpenSSL 3.0 was available). My understanding of the flag
-I/opt/local/libexec/openssl3/include -I/opt/local/libexec/openssl11//include
is that GCC will search OpenSSL headers in OpenSSL 3.0 before it even look at OpenSSL 1.1; because the header openssl/opensslv.h is available in OpenSSL 3.0, it stops there. ~I guess -I/opt/local/libexec/openssl3/include was added by ruby-install? If so, maybe we want a way for users to override that?~ I read the source code mentioned above, and it seems that ruby-install directly passes --with-opt-dir=/opt/local to ./configure.
I checked the generated ./configure, and noticed that --with-opt-dir actually accept a list of directories, separated by a colon. So I monkey-patched ruby/functions.sh like
opt_dir="${opt_dir_prefix:+$opt_dir_prefix:}$opt_dir"
and specified the OpenSSL 1.1 directory with $opt_dir_prefix as environment variable. This seems to work, and I now come across #394 (which is an unrelated issue).
Sorry, my opt_dir_prefix didn’t fix the issue. Now ext/openssl/mkmf.log says that it’s using command
gcc -I../../.ext/include/x86_64-darwin21 -I../.././include -I../.././ext/openssl -I/opt/local/libexec/openssl3/include -I/opt/local/libexec/openssl11/include -I/opt/local/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -g -O2 -pipe -c conftest.c
which has OpenSSL 3 before 1.1. This seems to come from ext/openssl/Makefile, which has
INCFLAGS = -I. -I$(arch_hdrdir) -I$(hdrdir) -I$(srcdir) -I/opt/local/libexec/openssl3/include
CPPFLAGS = -DHAVE_OPENSSL_SSL_H -I/opt/local/libexec/openssl11/include -I/opt/local/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT $(DEFS) $(cppflags)
I’m not sure how Ruby generates the Makefile, but I’ll dig deeper.
Bumping the Ruby OpenSSL default gem to version 3.0 would fix this for Ruby 2.6 and 2.7, but it'd mean dropping support in these Rubies for OpenSSL 1.0.1. Unfortunately that means it's a blocker, since these older Rubies are contemporary with OpenSSL 1.0.1, unless you toggle which default gem is shipped (considered in rvm/rvm#5209) based on whether the system provides OpenSSL 3. See https://bugs.ruby-lang.org/issues/18658
It's a shame it blocks install because all Ruby 2.6 and 2.7 need is a gem update openssl for this to be resolved.
I’m fine with Ruby 2.7 not supporting OpenSSL 3.0; this is pretty much expected. After reading the bug, I also understand their concern against bumping the built-in gem version, but both are beside my point.
What I’ve been asking for is to link the to the OpenSSL 1.1 from MacPorts. Being unable to specify OpenSSL path is the “bug” I’d like to report (and hopefully get resolved). I do feel like it could belong upstream.
I guess the way autoconf/./configure works is that --with-openssl-dir just adds the directory along with the --with-opt-dir directories. We could adjust the ./configure option building logic to pass in individual --with-___-dir options. This would allow overriding the default generated --with-___-dir options via the additional command-line arguments (ex: ruby-install ruby -- --with-___-dir ...).
@FranklinYu and do you still want the other dependencies to come from homebrew, or everything linked against what is installed by MacPorts? Specifying --package-manager port will force ruby-install to use MacPorts, even if homebrew is also installed. Additionally, I could add an RUBY_INSTALL_PACKAGE_MANAGER env variable to allow permanently configuring the desired package manager.
I want everything linked against MacPorts. I do have --package-manager port already, but Ruby ./configure finds OpenSSL 3 before 1.1 (we know why), which is the only reason why I wanted to explicitly specify its location.
We could adjust the
./configureoption building logic to pass in individual--with-___-diroptions.
I thought we already pass everything after -- to ./configure? This is my full invocation:
opt_dir_prefix=/opt/local/libexec/openssl11 ruby-install ruby 2.7 -- --with-openssl-dir=/opt/local/libexec/openssl11
The command ruby-install ruby 2.7 -- --with-openssl-dir=/opt/local/libexec/openssl11 would expand to:
./configure --prefix="$install_dir" --with-opt-dir="$opt_dir" --with-openssl-dir=/opt/local/libexec/openssl11
So I suspect the ./configure script is naively checking the --with-opt-dir first then checking --with-openssl-dir, or possibly somehow finding the homebrew openssl directory via some other means, and adding each directory to some kind of array variable. Perhaps if we used explicit --with-___-dir options for each dependency, the ./configure command would expand to this:
./configure --prefix="$install_dir" --with-openssl-dir="$(brew --prefix [email protected])" ... --with-openssl-dir=/opt/local/libexec/openssl11
I would expect that command to properly override the homebrew's openssl directory with your own; unless this is actually a bug in ruby's ./configure script.
It is not finding my Homebrew OpenSSL (I don’t install OpenSSL with Homebrew), but actually my OpenSSL 3.0 from MacPorts at /opt/local/{include,lib}. I did some digging, apparently it is using pkg-config to find a version of OpenSSL, and append it to $INCFLAGS. In addition, $INCFLAGS is before $CFLAGS:
https://github.com/ruby/ruby/blob/c9c2245c0a25176072e02db9254f0e0c84c805cd/lib/mkmf.rb#L524-L525
As a result, the one found by pkg-config will override the one specified by --with-openssl-dir.
Yes, you need to set PKG_CONFIG_PATH=myopenssl/lib/pkgconfig:$PKG_CONFIG_PATH as a workaround.
This is confusing and comes from openssl extconf.rb, could you please file an issue at https://bugs.ruby-lang.org/ or https://github.com/ruby/openssl? Not respecting --with-openssl-dir seems a clear bug.
In TruffleRuby we worked around this issue by setting PKG_CONFIG_PATH when the openssl dir is set (via $OPENSSL_PREFIX), which is what I already said in https://github.com/postmodern/ruby-install/issues/412#issuecomment-1006825955.
@eregon On my machine, setting $PKG_CONFIG_PATH helps, but it’s insufficient. I ran into the issue
Traceback (most recent call last):
15: from ./tool/rbinstall.rb:649:in `<main>'
14: from ./tool/rbinstall.rb:713:in `<module:RbInstall>'
13: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
12: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
11: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/installer.rb:11:in `<top (required)>'
10: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
9: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
8: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/package.rb:44:in `<top (required)>'
7: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
6: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
5: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/security.rb:12:in `<top (required)>'
4: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
3: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
2: from /Users/franklinyu/src/ruby-2.7.6/.ext/common/openssl.rb:13:in `<top (required)>'
1: from /Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require'
/Users/franklinyu/src/ruby-2.7.6/lib/rubygems/core_ext/kernel_require.rb:83:in `require': dlopen(/Users/franklinyu/src/ruby-2.7.6/.ext/x86_64-darwin21/openssl.bundle, 0x0009): symbol not found in flat namespace '_EVP_CIPHER_CTX_block_size' - /Users/franklinyu/src/ruby-2.7.6/.ext/x86_64-darwin21/openssl.bundle (LoadError)
The openssl.bundle turned out to be compiled with OpenSSL 1.1, but linked with OpenSSL 3. I managed to install Ruby 2.7 using both this $PKG_CONFIG_PATH trick and my monkey-patched $opt_dir_prefix mentioned above.
Do we know any place documenting the --with-openssl-dir option? I’m unsure whether I just imagined it myself, because Ruby ./configure --help doesn’t show it.
Ok, the --with-xxx-dir is implemented in MakeMakefile#dir_config():
https://github.com/ruby/ruby/blob/c9c2245c0a25176072e02db9254f0e0c84c805cd/lib/mkmf.rb#L1738-L1801
It is called here:
https://github.com/ruby/ruby/blob/c9c2245c0a25176072e02db9254f0e0c84c805cd/ext/openssl/extconf.rb#L17
So in ext/openssl/extconf.rb the find_openssl_library() should have found the OpenSSL 1.1 library. It didn’t, because ext/openssl/extconf.rb has
pkg_config_found = pkg_config("openssl") && have_header("openssl/ssl.h")
if !pkg_config_found && !find_openssl_library
raise "error"
end
which means that find_openssl_library() will be attempted only if pkg-config didn’t find it. In summary, the Ruby team might not consider this a “bug”, but --with-xxx-dir=yyy actually means “you may (or may not) also find xxx in yyy”, instead of “you must use the xxx located at yyy” (like I thought).
(Phew, extensive source-code digging.)
In summary, the Ruby team might not consider this a “bug”, but
--with-xxx-dir=yyyactually means “you may (or may not) also find xxx in yyy”, instead of “you must use the xxx located at yyy” (like I thought).
I think 99+% of people would think --with-openssl-dir=... means "use that one", so IMHO a bug it doesn't respect that, could you report it?
Also the fact it's possible to pick headers from one version and lib from another seems also sketchy, probably a bad interaction between these 2 ways to find openssl (or 3 counting --with-opt-dir).
Sorry, didn’t had time to do it earlier. Here it is:
https://bugs.ruby-lang.org/issues/18763
Took me a while to get this working, but the following worked for me on MacOS after I did an upgrade to Monterey. Thanks @postmodern and @eregon, it was a combination of your above posts.
brew install [email protected]
PKG_CONFIG_PATH=/usr/local/opt/[email protected]/lib/pkgconfig:$PKG_CONFIG_PATH ruby-install ruby 2.7 -- --with-openssl-dir=/usr/local/opt/[email protected]
The easiest way to install CRuby < 3.1 with a system OpenSSL 3/OpenSSL 3 installed is to use ruby-build which handles this stuff for you out of the box.
The easiest way to install CRuby < 3.1 with a system OpenSSL 3/OpenSSL 3 installed is to use ruby-build which handles this stuff for you out of the box.
How will ruby-build handle security advisories for the copy of openssl-1.1 that ruby-build compiles from source?
How will ruby-build handle security advisories for the copy of openssl-1.1 that ruby-build compiles from source?
Similar to how one would handle security advisories for a Ruby built from source, i.e., rebuild that Ruby (and its openssl) when there is a new CVE.
How will ruby-build handle security advisories for the copy of openssl-1.1 that ruby-build compiles from source?
Similar to how one would handle security advisories for a Ruby built from source, i.e., rebuild that Ruby (and its openssl) when there is a new CVE.
This is not exactly correct and slightly misleading. There is a difference between simply installing upstream software, without modification from source, and explicitly deciding for the end-user that specific versions of dependencies will be installed from source. ruby-build is explicitly deciding which version of openssl to install (and where it is installed), and thus is taking on the responsibility of having to update that version, notify it's users which rubies were effected, how to update them, etc, anytime a new security advisory is published for that version of openssl. However, if a ruby is built from source, without --enable-static, then it's openssl bindings dynamically link to the libopenssl.so shared object library typically installed by the package manager. This allows the package manager to update libopenssl.so without having to recompile any other software on the system which dynamically links to it; provided the update does not break the ABI (Application Binary Interface), which no security update should ever do. This allows the package manager to handle the responsibility of safely updating dependencies whenever a security update is released for them.
Somehow you are not mentioning the same awareness is needed when building Ruby from source (e.g. with ruby-install/ruby-build/RVM), if there is a Ruby CVE the fix is to install a newer release of that Ruby. For ruby-build, if the system OpenSSL cannot be used for that Ruby, additionally when there is a OpenSSL CVE one needs to recompile the Rubies depending on it. It's nothing hidden, the ruby-build output shows it installs OpenSSL and AFAIK very few users ever complained about it. And anyway, isn't that better than not being to build Ruby < 3.1 on systems with a system OpenSSL 3? Compiling from source & configuring manually is no better obviously, and much easier to get wrong (as shown above, it's not easy). I guess it's the biggest difference between ruby-install and ruby-build, ruby-install only works in some happy cases where the system OpenSSL luckily matches what that Ruby needs, ruby-build works for a lot more Ruby versions but that means compiling its own OpenSSL/recompiling on CVEs and asking the user to install other dependencies manually.
On macOS it's slightly different because it's fairly easy to install multiple "system OpenSSL" with Homebrew, although that also typically ends up causing problems. On Linux it's simple, there is one system OpenSSL version and no other (i.e. all system packages depend on a specific OpenSSL version so even if there another OpenSSL package it's unusable in practice), hence there is no choice to build a custom OpenSSL version for older Rubies on Linux.
provided the update does not break the ABI (Application Binary Interface), which no security update should ever do.
A big assumption, OpenSSL is historically rather bad at ABI, and I've read many times how updating Homebrew OpenSSL breaks Rubies compiled against it (maybe it got better, unsure).
Somehow you are not mentioning the same awareness is needed when building Ruby from source (e.g. with ruby-install/ruby-build/RVM), if there is a Ruby CVE the fix is to install a newer release of that Ruby.
This is true, but we are not discussing potential vulnerabilities in CRuby itself, but instead potential vulnerabilities in one of it's dependencies, openssl, which CRuby links against. This is a different scenario.
And anyway, isn't that better than not being to build Ruby < 3.1 on systems with a system OpenSSL 3?
Yes, it is good that there's a workaround for this issue. However, it is also creating a potential problem and additional work should a vulnerability be announced for the version of openssl which ruby-build is explicitly using. I would recommend having a security response plan ready for when the next openssl vulnerability is announced (advisory template, list of effected versions, instructions for users on how to identify if a ruby version was compiled against openssl compiled by ruby-build, etc).
I guess it's the biggest difference between ruby-install and ruby-build, ruby-install only works in some happy cases where the system OpenSSL luckily matches what that Ruby needs,
It is ruby-install's policy to not workaround upstream bugs (be them ruby related or distro/OS issues). If a CRuby version does not support a newer/older version of openssl, then ruby-core needs to release a new version with updated openssl bindings; or provide a way of updating the bundled openssl gem. If we continue to add workarounds and patches for upstream issues, this will allow upstream to become lazy or disconnected from users; this has happened in the past. This may not be the most popular or convenient stance, but it keeps upstream accountable.
A big assumption, OpenSSL is historically rather bad at ABI, and I've read many times how updating Homebrew OpenSSL breaks Rubies compiled against it (maybe it got better, unsure).
If an OpenSSL security update does break ABI, then yes we should instruct users to also re-compile the ruby. If an OpenSSL security update does not break ABI, than updating the openssl library is all that is necessary.
It is ruby-install's policy to not workaround upstream bugs (be them ruby related or distro/OS issues). This may not be the most popular or convenient stance, but it keeps upstream accountable.
Agreed, this makes a lot of sense, I think both approaches are useful and have their trade-offs. Hence my first comment was meant as "if ruby-install doesn't work due to openssl issues, an easy solution/workaround is to use ruby-build".
If a CRuby version does not support a newer/older version of openssl, then ruby-core needs to release a new version with updated openssl bindings; or provide a way of updating the bundled openssl gem. If we continue to add workarounds and patches for upstream issues, this will allow upstream to become lazy or disconnected from users; this has happened in the past.
I definitely agree on that, unfortunately CRuby doesn't seem to want to take that step, after asking repeatedly (https://bugs.ruby-lang.org/issues/18658).
This solution definitely works for MacOS Catalina & Macports & RVM & Ruby 2.X.Y < 3.0:
PKG_CONFIG_PATH=/opt/local/libexec/openssl11/lib/pkgconfig:$PKG_CONFIG_PATH rvm reinstall ruby-2.6.3 -- --with-openssl-dir=/opt/local/libexec/openssl11
This solution definitely works for MacOS Catalina & Macports & RVM & Ruby 2.X.Y < 3.0:
PKG_CONFIG_PATH=/opt/local/libexec/openssl11/lib/pkgconfig:$PKG_CONFIG_PATH rvm reinstall ruby-2.6.3 -- --with-openssl-dir=/opt/local/libexec/openssl11
I had a problem installing the new ruby version for react native 70+, and your comment solved my problem. I only had to change the open SSL directory to /opt/homebrew/opt/[email protected] because I'm using homebrew.
My setup is M2 MacOS Ventura 13.2.1
I have recently added some logic to dynamically install the [email protected] or openssl@3 homebrew package depending on the version of Ruby. See a37e332e055da91db5fb053a3d94ebb0f219043d I also duplicated the logic for MacPorts openssl11 and openssl3 packages in commit 22987ed5fd7ed4bac917a3904d7b700bc91543d7. This will be released shortly in ruby-install 0.9.3.
Also, ruby-install 0.9.3 now specifies --with-openssl-dir instead of just
--with-opt-dir, which apparently Ruby's configure script treats with more
precedence, and should prevent it from using another detected instance of
openssl on the system.
However, this comes with a few caveats:
- This is risky, because openssl-1.1 has reached End-of-Life. It will not receive anymore security updates. While I understand people have legacy apps they must maintain, running old unmaintained versions of software is risky. I strongly recommend that people upgrade to at least Ruby 3.1, which uses openssl-3.0.
- This only works for homebrew and MacPorts users. Linux and other BSD package managers only provide one version of openssl, and that's most likely openssl-3.0, as openssl-1.1 has reached End-of-Life.
- I will eventually remove this complex logic after Ruby 3.0 reaches End-of-Life, which was the last Ruby version to use openssl-1.1. After that point, if users still need to install Ruby 2.7 or 3.0, they will have to specify
-- --with-openssl-dir=/path/to/[email protected]themselves.
This should finally fix this issue. Comment or reopen this issue if you still have issues with ruby-install 0.9.3.
Took me a while to get this working, but the following worked for me on MacOS after I did an upgrade to Monterey. Thanks @postmodern and @eregon, it was a combination of your above posts.
brew install [email protected] PKG_CONFIG_PATH=/usr/local/opt/[email protected]/lib/pkgconfig:$PKG_CONFIG_PATH ruby-install ruby 2.7 -- --with-openssl-dir=/usr/local/opt/[email protected]
You save my day!
Only a few changes for M1 Mac /usr/local/opt/[email protected] to /opt/homebrew/opt/[email protected]
brew install [email protected]
PKG_CONFIG_PATH=/opt/homebrew/opt/[email protected]/lib/pkgconfig:$PKG_CONFIG_PATH ruby-install ruby 2.7.5 -- --with-openssl-dir=/opt/homebrew/opt/[email protected]
@komendantaa ruby-install 0.9.3 should fix this and will automatically switch between [email protected] and openssl@3. Have you tested ruby-install 0.9.3?
Here’s the very strange variation that seemed to work for me—no idea why my path is slightly weird! I may need this next time I install Ruby (hello there future me!).
PKG_CONFIG_PATH=/opt/homebrew/Cellar/[email protected]/1.1.1w/lib/pkgconfig:$PKG_CONFIG_PATH ruby-install ruby 2.7.4 -- --with-openssl-dir=/opt/homebrew/Cellar/[email protected]/1.1.1w
You save my day! Only a few changes for M1 Mac
/usr/local/opt/[email protected]to/opt/homebrew/opt/[email protected]brew install [email protected] PKG_CONFIG_PATH=/opt/homebrew/opt/[email protected]/lib/pkgconfig:$PKG_CONFIG_PATH ruby-install ruby 2.7.5 -- --with-openssl-dir=/opt/homebrew/opt/[email protected]
FWIW I was installing this version to support using https://github.com/github/pages-gem locally to build GitHub Pages sites using Jekyll. Once I successfully installed Ruby 2.7.4 I had to delete the existing Gemfile.lock before I could successfully run bundle install, as it was failing with:
An error occurred while installing ffi (1.11.1), and Bundler cannot continue.
Make sure that `gem install ffi -v '1.11.1' --source 'https://rubygems.org/'` succeeds before bundling.
