pynsq icon indicating copy to clipboard operation
pynsq copied to clipboard

Reader keeps querying nsqlookupd after `reader.close()` if it's called too quickly

Open dettonijr opened this issue 3 years ago • 1 comments

import nsq
import tornado

reader = nsq.Reader(
    lookupd_http_addresses=["localhost:4161"],
    topic="topic",
    channel="channel",
    message_handler=lambda msg: None,
    lookupd_poll_interval=5,
    lookupd_poll_jitter=1.0,
)

ioloop = tornado.ioloop.IOLoop.current()

# Very quick, but this needs to run after reader._run
# EDIT: Ignore the following 2 lines. The reason it needs to be after _run is because it needs to schedule the callback that will trigger the bug. Thinking now, I guess "add_callback" should actually reproduce, but I remember it didn't.
# Because there's another bug in `reader.close` if self.redist_periodic is None
# In theory we should be able to just add_callback here without call_later, or even just call `reader.close()`
ioloop.call_later(0.1, reader.close)

# Callback just to make the program end eventually, not necessary to reproduce the bug
ioloop.call_later(300, ioloop.stop)
nsq.run()

The example above will (most of the times) keep querying nsqlookupd every 5 seconds, even after reader.close was called

This seems to be because this delayed callback never gets canceled https://github.com/nsqio/pynsq/blob/master/nsq/reader.py#L263

Note: Because of the random() in the delay, it "works sometimes" when random is a very small number

dettonijr avatar Dec 31 '21 10:12 dettonijr

Yep, good catch. Are you interested in opening a PR to fix this?

mreiferson avatar Jan 16 '22 19:01 mreiferson