Connection to Lets Encrypt secured server fails

[corny]

OMG this library uses a built-in list of CA certificates from 2015: :hankey: :man_facepalming:

##
## Bundle of CA Root Certificates
##
## Certificate data from Mozilla as of: Wed Oct 28 04:12:04 2015

My quick fix for Debian/Ubuntu:

ln -sf /etc/ssl/certs/ca-certificates.crt  $rubydir/2.7.0/gems/httpclient-2.8.3/lib/httpclient/cacert.pem

[jhollinger]

Yeah, pretty crazy that it's doing that. Our fix was the following mokeypatch:

require 'httpclient'

class HTTPClient
  alias original_initialize initialize

  def initialize(*args, &block)
    original_initialize(*args, &block)
    # Force use of the default system CA certs (instead of the 6 year old bundled ones)
    @session_manager&.ssl_config&.set_default_paths
  end
end

[jessebs]

Building off of @jhollinger, just calling client.ssl_config.set_default_paths without overriding the constructor seems to work

client = HTTPClient.new
client.ssl_config.set_default_paths

[jdongelmans]

Just an FYI: this gem is a dependency for Rack::OAuth2 and broke our production, so we added an initializer basically doing the same as mentioned above:

Rack::OAuth2.http_config do |c|
  c.ssl_config.clear_cert_store
  c.ssl_config.cert_store.set_default_paths
end

[willtcarey]

Our application also uses Webfinger in our OAuth stack. The process of getting it to use the system certs was similar to Rack::OAuth2.

WebFinger.http_config do |c|
  c.ssl_config.clear_cert_store
  c.ssl_config.cert_store.set_default_paths
end

[jamesyale]

Thanks all, based on @jhollinger 's suggestion we hacked the library directly: https://github.com/livelink/httpclient/commit/c1b97a7567e75c31c99fb502b10bfb975d921196#diff-2a6765023b9beb8295399d5c04286045360ce90aabfffead2dba5c2c3ef71173

[essjayhch]

Just FYI, the monkey patch is an ruby 2.3+ feature set.

Not that I'm suggesting httpclient should specifically support ancient versions of ruby out there, it is worth noting that this gem has been fundamental since the days or rails 2, which love it or hate it still exist out there in the wild.

For those coming across this and looking to fix a certificate error on an unsupported version of ruby, the problem you will be tripping over will be the &. (safe operator) littered through the patch.

Yeah, pretty crazy that it's doing that. Our fix was the following mokeypatch:

require 'httpclient'

class HTTPClient
  alias original_initialize initialize

  def initialize(*args, &block)
    original_initialize(*args, &block)
    # Force use of the default system CA certs (instead of the 6 year old bundled ones)
    @session_manager&.ssl_config&.set_default_paths
  end
end

Thinking about this (as we've had to patch some pre ruby 2.3 legacy code), you shouldn't need the safe operator, as the @ssl_config attribute should already be setup by the time you get to the end of the constructor anyway.

[heinrichhanekom]

We had the same thing. Thanks for the fixes:

OpenIDConnect.http_config do |c|
  c.ssl_config.clear_cert_store
  c.ssl_config.cert_store.set_default_paths
end

Rack::OAuth2.http_config do |c|
  c.ssl_config.clear_cert_store
  c.ssl_config.cert_store.set_default_paths
end

WebFinger.http_config do |c|
  c.ssl_config.clear_cert_store
  c.ssl_config.cert_store.set_default_paths
end

[jhollinger]

Is this lib even actively maintained? There's not much commit activity at all. I've rarely seen anyone use httpclient directly in their apps; usually it's a transitive dependency (openid_connect, rack-oauth2, etc). Could we put pressure on those projects to reduce their dependencies? I'll grant that the stdlib's HTTP client doesn't have the cleanest API, but it works just fine. The community has kind of "left-padded" itself, here, IMHO.

[gareth]

It might have got hidden in the references above, but thanks to @nov there are now new versions of webfinger, rack-oauth2, fb_graph2, swd and openid_connect that have these fixes incorporated.

[jfederico]

Something else brought me to here, but yeah, this is pretty bad.

It would be super hard to update properly. The dependency to version 2.8.3 comes from google-apis-core (0.3.0), but there are some other gems depending on even older versions. We are using omniauth_openid_connect 0.3.5 (June 7, 2020), which uses openid_connect ~> 1.1 (March 23, 2017), which uses webfinger >= 1.0.1 (December 22, 2014), which uses httpclient >= 2.4 (June 8, 2014).

The patch contributed here #2930 seems to be the best choice right now. Although it may need to be extended.

We are looking into it

[RiverHeart]

This appears to have affected apt-listbugs as well, which in our case was preventing updates from going through. Fix is the same as the original, but uses the path to the system version /usr/lib/ruby/vendor_ruby/httpclient/cacert.pem

Thread first indicating the issue with apt-listbugs https://www.mail-archive.com/[email protected]/msg1822839.html

[mbeijen]

This issue was fixed in debians version of the package here, and like this: https://salsa.debian.org/ruby-team/ruby-httpclient/-/commit/542849f1b60e9c0cd24c328ad710b2a94cb42729

[jrochkind]

@nahi no action on this in over ayear may lead me to abandon HTTPClient, when I otherwise rather like it. :(