RecursionError if many calls were waiting for free connection

[IlyaSkriblovsky]

This code either hangs (py2) or crashes with RecursionError (py3):

from __future__ import print_function

@defer.inlineCallbacks
def main():
    redis = txredisapi.lazyConnection()
    redis.set('c', 0)

    yield defer.gatherResults(
        redis.incr('c').addCallback(print)
        for i in range(0, 1000)
    )

Each non-blocking call to Redis does connectionQueue.get() to get connection and then immediately does connectionQueue.put() because it's non-blocking. If there are other pending calls, DeferredQueue.put() immediately passes released connection to next pending call in same execution stack. This next call in its turn immediately calls connectionQueue.put() again.

So, if many queries were waiting for free connection (because either connection wasn't made yet like in code above or it is blocked by transaction), when connection is ready, all these queries will be processed recursively in a single execution stack. Default Python's recursion limit of 1000 gets exhausted after processing 73–74 queries.

The dumb solution: replace self._factory.connectionQueue.put(connection) (txredisapi.py:1864) by reactor.callLater(0, self._factory.connectionQueue.put, connection) This solves the issue, but is very ugly.

[IlyaSkriblovsky]

Much nicer solution is in #107