boto icon indicating copy to clipboard operation
boto copied to clipboard

Fix HostConnectionPool quadratic runtime

Open carsonip opened this issue 7 years ago • 3 comments

list.pop(0) is O(N) such that get() is quadratic. Use deque and six.moves.range to keep get() always O(N). Add tests to HostConnectionPool.

carsonip avatar Mar 08 '19 07:03 carsonip

Test fails due to the previous SNI commit.

carsonip avatar Mar 08 '19 07:03 carsonip

Benchmark script for this commit:

import time
from boto.connection import HostConnectionPool


def f():
    p = HostConnectionPool()
    count = 10000
    for _ in xrange(count):
        p.put(None)
    for _ in xrange(count):
        assert p.size() == count - _
        p.get()


t = time.time()
f()
print(time.time() - t)

Before fix: 0.27458691597 After fix: 0.0182430744171

carsonip avatar Mar 08 '19 07:03 carsonip

Test fails due to the previous SNI commit.

@carsonip looks like we are waiting on #3844 to be merged to fix python2.6 due to dependency issues :(

jayninja avatar Mar 08 '19 17:03 jayninja