Connection pool optimization: move socket polling from expiry checks to connection usage

[MarkusSintonen]

Summary

(Split from original PR here https://github.com/encode/httpcore/pull/924 but this is more refined)

Connection pool implementation is heavily doing socket polling via AsyncConnectionInterface.has_expired(). This check in turn does AsyncNetworkStream.get_extra_info("is_readable") which finally does select.poll(). This check quickly starts to add up when doing many requests through httpcore with great concurrency. Many connection sockets in the pool are constantly polled over and over again when doing requests and also when request closes.

This PR instead moved the socket polling into AsyncHTTP11Connection.handle_async_request. Here the readable-IDLE connection raises ConnectionNotAvailable which makes the request to choose a next connection from the pool. This causes lot less socket polling as now its not done every single time for all connections when pool connections are assigned. Broken connections are still properly maintained as they are still removed from the pool.

This approach is very similar on how urllib3 is validating the health of the connection coming from the pool. (This check finally uses wait_for_read which uses similar socket polling as httpcore)

Adds some previously missing tests.

Benchmark shows how the heavy socket logic causes over 4.6x overhead.

In master with async requests:

PR with async:

With synchronous code the overhead is not as large. There the connection pools socket polling causes 1.6x overhead (maybe the overhead is lower due to sync concurrency using threads where the socket polling IO is not in GIL). Rest of overhead in sync is coming from the connection pools maintenance loops that is fixed here.

In master with sync code:

PR with sync:

Trio-based backend is also affected by this. There the overhead is at similar levels of about 5x as was seen here.

Checklist

• [x] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• [x] I've updated the documentation accordingly.