Http client should restrict the amount of connections opened

[reactormonk]

When running mapConcurrently httpLBS requests I get loads of ConnectionFailure Network.BSD.getProtocolByName: does not exist (no such protocol name: tcp)), requiring some additional code to restrict the amount of connections being created.

[snoyberg]

The short answer is "don't do that." The connection manager is designed for sharing connections, not preventing a DoS attack. I'm not inherently opposed to including some level of rate limiting in the library, but I have no plans to implement it myself at this time.

[andrewthad]

I also think something like this would be nice to include in http-client.

[creichert]

Are you certain all the "no such protocol" errors have anything to do w/ the rate at which your making requests? I had the same issue very recently and it did seem to happen when making a burst of requests rather than a one-off.

I mentioned this in another thread, but I get these errors when I'm using a docker container w/o netbase installed or a weird /etc/hosts.

Here's my comment from #292:

I believe @exarkun is right. I had this issue recently and had to install netbase. What's odd is that the failures seemed intermittent.

If the issue persists after installing netbase, check the /etc/hosts file of the to make sure it's sane. I believe at minimum:

$ cat /etc/hosts
127.0.0.1       localhost
::1             localhost ip6-localhost ip6-loopback
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters

[reactormonk]

Both issues produce the same error messages - in one case the file isn't there, in the other it can't be opened because there's no more file descriptors.

[neongreen]

I need rate limiting in https://github.com/wireapp/wire-server so I implemented something (without even trying to compile it, of course): https://gist.github.com/19dd06528d6a7e5b776616328547fe42. I plan to make it work on Monday (ish), but if anybody familiar with http-client wants to briefly look at my implementation and suggest ways in which it will inevitably fail, that would be more than welcome.