pool icon indicating copy to clipboard operation
pool copied to clipboard

Program hangs with v0.3

Open ysbaddaden opened this issue 1 year ago • 3 comments

While investigating an issue I found out that the following stress test will eventually hang (without MT):

require "pool/connection"

class Foo
end

pool = ConnectionPool(Foo).new(capacity: 5, timeout: 50.milliseconds) { Foo.new }

10_000.times do |i|
  spawn do
    loop do
      pool.connection
      sleep rand(1..10).milliseconds
      pool.release
    end
  end
end

sleep

The program will print a lot of IO::TimeoutError exceptions then hang. A quick gdb session shows that it's waiting on epoll_status, that is Crystal is waiting for events, forever. Weirdly, I'd expect the select timeout to keep raising, but nope it doesn't even reach that stage.

Maybe an unchecked mutex? or both mutexes not playing nicely?

Related to https://github.com/stefanwille/crystal-redis/issues/134

ysbaddaden avatar Mar 24 '23 15:03 ysbaddaden

what if remove timeout at all? and timeout would be handled on user side:

conn = pool.checkout?
unless conn
  sleep 5.0
  conn = pool.checkout?
end

upd: bad idea, because it would be too big all time.

kostya avatar Mar 25 '23 11:03 kostya

Yeah, we want early resume.

To be honest, I don't know what's going awry. This line is wrong (it blocks until there is a value pushed to the channel or the channel is closed), but removing the whole block doesn't seem to fix the issue:

https://github.com/ysbaddaden/pool/blob/main/src/pool.cr#L57-L59

ysbaddaden avatar Mar 26 '23 12:03 ysbaddaden

Also @pool should be a Deque instead of an Array. Or maybe @pool shouldn't exist and all connections should always be pushed to the channel? Then I doubt Pool(T) brings much value over a Channel(T) + select timeout() anymore?

ysbaddaden avatar Mar 26 '23 13:03 ysbaddaden