taskiq icon indicating copy to clipboard operation
taskiq copied to clipboard

Scheduler log spam: broker mismatch between `AsyncSharedBroker` and `Redis ListQueueBroker`

Open Bohdan-Ilchyshyn opened this issue 7 months ago • 1 comments

While running a Taskiq scheduler that uses taskiq_redis.redis_broker.ListQueueBroker, the worker constantly prints warnings like the following for every scheduled tick:

Broker for taskiq_pipelines.shared.filter_tasks <taskiq.brokers.shared_broker.AsyncSharedBroker object at 0xfef8d4d6b8c0> doesn't match scheduler's broker <taskiq_redis.redis_broker.ListQueueBroker object at 0xfef8d42606e0>
Broker for taskiq_pipelines.shared.wait_tasks   <taskiq.brokers.shared_broker.AsyncSharedBroker object at 0xfef8d4d6b8c0> doesn't match scheduler's broker <taskiq_redis.redis_broker.ListQueueBroker object at 0xfef8d42606e0>

Tasks still run, but the log noise is severe and suggests something is mis-configured

My code:

REDIS_CONNECTION_KWARGS = {
    'retry': Retry(ExponentialBackoff(), retries=3),
    'retry_on_error': [BusyLoadingError, ConnectionError, TimeoutError],
    'max_connection_pool_size': 100,
}

result_backend = RedisAsyncResultBackend(
    redis_url=settings.redis_url, **REDIS_CONNECTION_KWARGS
)
broker = ListQueueBroker(
    url=settings.redis_url, queue_name='taskiq-default', **REDIS_CONNECTION_KWARGS
)
broker.with_result_backend(result_backend=result_backend)
broker.with_middlewares(LoggingMiddleware(), PipelineMiddleware())
scheduler = TaskiqScheduler(
    broker=broker,
    sources=[LabelScheduleSource(broker)],
)

taskiq_fastapi.init(
    broker,
    'src.web.application:get_app',
)

Lib versions:

taskiq-dependencies==1.5.7 ; python_version == "3.12"
taskiq-fastapi==0.3.4 ; python_version == "3.12"
taskiq-pipelines==0.1.4 ; python_version == "3.12"
taskiq-redis==1.0.2 ; python_version == "3.12"
taskiq==0.11.13 ; python_version == "3.12"

Bohdan-Ilchyshyn avatar Apr 23 '25 09:04 Bohdan-Ilchyshyn