Request IDs not freed after fetching the resource

[jsdalton]

Due to a recently introduced change (33fb966197f14772e750a167638f1cb49d1f3165), it would appear that that request IDs are no longer freed after a resource has been fetched.

The end result is that the cache of request IDs grows to its max size after about 64k DNS resolution requests, and the program is stuck in an infinite loop thereafter.

Details

Allocated request IDs are cleaned up after each request via DNS.free_request_id, which is called on each sender. However, 33fb966197f14772e750a167638f1cb49d1f3165 introduces a change which deletes the sender after the request is made: https://github.com/ruby/resolv/blob/f85979f5c5ff70c7188d27f407f70a3ce6fc5f09/lib/resolv.rb#L708-L710

This means, in the #close method, @senders is always empty for most Requesters: https://github.com/ruby/resolv/blob/f85979f5c5ff70c7188d27f407f70a3ce6fc5f09/lib/resolv.rb#L782-L792

Request ids are thus not getting deallocated.

The larger problem is that DNS.allocate_request_id is only capable of assigning a maximum of 64k request ids. Once the hash it uses to store these is fully populated, it goes into an infinite loop searching for a free slot that will never be filled.

The end result for users of this module is that after approximately 64k DNS requests, the program will halt execution in the middle of this while loop: https://github.com/ruby/resolv/blob/f85979f5c5ff70c7188d27f407f70a3ce6fc5f09/lib/resolv.rb#L624-L626

Since the DNS::RequestID hash is stored as a class constant, instatiating a new instance of DNS will not solve the problem. The current workaround would be to manually flush values from the RequestID

Reproducing

It's fairly easy to demonstrate that the DNS::RequestID hash fills without ever being cleaned up.

The following script shows the length of the request ID cache growing linearly with each request. This script will also max at out 65536 (with some slow downs at higher numbers as it searches for available slots in the hash):

require 'resolv'

puts RUBY_VERSION
puts Resolv::DNS::RequestID
resolver = Resolv::DNS.new
domain = 'example.com'

i = 1
loop do
  resolver.getresources(domain, Resolv::DNS::Resource::IN::A)
  puts Resolv::DNS::RequestID.values.first.length
  i += 1
end

$ ruby thread_test.rb
2.7.3
{}
1
2
3
4
5
# ...

Impact

We recently ran into this issue in a long running script, which needs to re-resolve DNS names frequently. The script was halting execution and we could not figure out why. Debugging led us to the while loop in this module and to the problem described above.

Let me know if you have questions or difficulties reproducing the issue.

[stevenharman]

I think we can close this as of #9, yeah?