AsyncStreamingClient 429 error on reconnect from uncaught exceptions

[acarasimon96]

First, thank you so much for (finally) adding an asynchronous Twitter API v2 streaming client in Tweepy v4.10. It works great and I like the fact that it can disconnect immediately rather than waiting for the next response from Twitter. However, I ran into a strange bug with this new implementation.

I subclassed AsyncStreamingClient to make it run forever until the user pressed Ctrl+C instead of exiting on its own when an uncaught exception occurred. Every time the client reconnects after the unhandled exception, I would get a 429 error for about 30 seconds before it would connect to the stream endpoint just fine. But the weird part is when I manually shut down my program and start it back up within those 30 seconds, the client connects successfully on the first try.

Here's the code I used to reproduce this scenario:

import asyncio
import logging
import sys
from tweepy.asynchronous.streaming import AsyncStreamingClient

BEARER_TOKEN = "yourtokenhere"


class RunForeverClient(AsyncStreamingClient):
    async def on_keep_alive(self):
        await super().on_keep_alive()
        # This will be triggered on the first keepalive signal once connected
        raise Exception("Uncaught exception")

    def run_forever(self) -> asyncio.Task:
        async def task():
            while True:
                await self.filter()  # or self.sample()
                if sys.exc_info()[0] == KeyboardInterrupt:
                    break
        return asyncio.create_task(task())


async def main():
    client = RunForeverClient(BEARER_TOKEN)
    await client.run_forever()


if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    try:
        asyncio.run(main())
    except KeyboardInterrupt:
        pass

[SirBernardPhilip]

I have the same issue, I would like to be able to keep the stream running when any exception occurs and I'm struggling with it as well

[Arctek]

I believe this is caused by aiohttp not closing the underlying connection - as the 429 error occurs when you try and connect concurrently to the streaming endpoint.

Though I'm not familiar enough with aiohttp to dig any deeper as to what exactly is causing it. FYI I'm on windows at the moment, so not sure if it has something to do with the underlying transport layer or just a general issue.

[nono-london]

Sorry to jump on this thread, but I am trying to updgrade my async streaming app to V2, and I can't find any example anywhere.

Could you please to point to any website to get started. tx

[Harmon758]

@nono-london That's unrelated to this issue.

You're likely encountering a 409 error due to a conflict in your rules. The latest development version of Tweepy now logs the specific error message, which you can probably use to determine the exact issue.

[Harmon758]

Thanks for the bug report and the reproducer. This took me a bit to wrangle, but it seems like this was due to lingering open sockets from closed SSL transports.