Pool Checkout - Timeout Required

[nerdyworm]

Good Morning!

This may be some other issue, however, it is completely possible for the checkout call to cause an entire system to hang indefinitely until the process is killed.

https://github.com/benoitc/hackney/blob/master/src/hackney_pool.erl#L66

[benoitc]

@nerdyworm 👍 if you have some time for it don't hesitate to send me a PR for it, otherwise I will do that sometimes this week

[ctrlShiftBryan]

@benoitc How did you envision fixing this? I've run into this issue because :hackney.body/1 is not called until the pool maxes out and things never get checked back in. Could you add a timeout to the amount of time something is not checked back in or is that best left for a pool handler?

[ctrlShiftBryan]

I ended up just respecting the connect_timeout option

[phylake]

Bump. By default hackney can only make 50 requests before it blocks completely.

[phylake]

That is to say this is a non-starter since we only get to 51

lists:foreach(fun(I) ->
  timer:sleep(500),
  io:format("req ~p~n",[I]),
  hackney:get("https://google.com")
end, lists:seq(1, 100)).

[benoitc]

@phylake you need to read the body. If not the socket won't be released to the pool.

Try the following:

lists:foreach(fun(I) ->
  timer:sleep(500),
  io:format("req ~p~n",[I]),
  {ok, _Status, _Headers, _Body} = hackney:get("https://google.com", [], <<>>, [with_body])
end, lists:seq(1, 100)).

[phylake]

The issue is present for post (and presumably put) as well which is how we originally discovered it

[phylake]

@benoitc do you consider the default get behavior a bug? Not all GETs have a body (e.g. on a 304)

[benoitc]

@phylake same for all connections. You need to read the response in any case to release the socket. This is how it works currently to prevent any resource consumption. (you can by pass the pool if you need to also).

304 and such cases are already covered :)

[benoitc]

@phylake late answer, but yes get should not accept a body. It will need some changes in the way CRUD calls are handled. I will take care about newxt release.