httprb
Remove Timeout.timeout
Timeout.timeout is not thread-safe and can interrupt any line, anywhere. Using native timeouts on the system level, or using IO.select is recommended for sockets. HTTP.rb already uses the right thing for read/write timeouts, but not connect.
I was able to use a better timeout for DNS resolving by overriding the socket class and using Resolv, however, that still leaves the opening of the actual socket, and the SSL handshake.
What can I do to avoid Timeout.timeout while not exposing myself to unreachable hosts taking infinitely long during the connect phase? I need to use HTTP.rb in a connection pool.
This is the sort of problem I was trying to address in Socketry, which supports multiple different types of resolvers for particularly these sorts of problematic cases:
https://github.com/socketry/socketry/tree/master/lib/socketry/resolver
The problem is Resolv is kind of bad, and will be missing a lot of the functionality in the system resolver, so it's important to have both.
There was a WIP PR to switch all of the I/O in the library over to Socketry, but I never completed it: https://github.com/httprb/http/pull/377
Alright, I have attempted to replace TCPSocket.open with something that calls Socket#connect_nonblocking with IO.select instead, but honestly, I am way out of my depth with this stuff, could you please take a look at this snippet:
https://github.com/tootsuite/mastodon/blob/43269ce9dbc3a93bd5bad4bf75d6e1e9ddfc1980/app/lib/request.rb#L177-L225
And tell me if this is complete garbage or not? It seems to work, and I seem to be able to both get normal non-timing out responses normally, and it seems to time out after 5 seconds on blackhole.webpagetest.org as it should (Mind that I monkey-patch the HTTP::Timeout::PerOperation class to remove Timeout.timeout call as well)
That's the basic idea, yes. See also:
https://github.com/socketry/socketry/blob/master/lib/socketry/resolver/resolv.rb
As an alternative to a wholesale move to Socketry, instead http.rb could just use its resolvers, and then it would be possible to do something like HTTP.dns_resolver(Socketry::Resolver::Resolv).
Did you guys comes to any stable conclusion on this?
My own solution lives in tootsuite/mastodon#10353