Issue starting worker with PrometheusMiddleware

[nickdichev-firework]

trafficstars

Hi, thanks for the awesome library. I am trying to add the PrometheusMiddleware to an application, however, I am seeing some strange behavior when starting the worker(s).

taskiq version: 0.8.2 (although I observed the same behavior on 0.8.1) prometheus-client version: 0.17.1 (although I observed the same behavior on 0.16.0)

I have defined my (redis) broker as follows:

broker = (
    ListQueueBroker(url=REDIS_URL, queue_name="nebula_tasks")
    .with_result_backend(redis_async_result)
    .with_id_generator(task_id_generator)
    .with_middlewares(
        PrometheusMiddleware(server_addr="0.0.0.0", server_port=9000),
    )
)

When I attempt to start my worker with default worker count I see the following trace (debug log enabled): pipenv run taskiq worker --fs-discover --workers 2 --log-level DEBUG src.nebula.broker:broker https://gist.github.com/nickdichev-firework/5b420be1d014f8fe925dcb63b771f5bd

When I attempt to start the worker with only one worker I see the following trace (debug log enabled): pipenv run taskiq worker --fs-discover --workers 1 --log-level DEBUG src.nebula.broker:broker https://gist.github.com/nickdichev-firework/36adf01a3ba11e22bf2c17766f61282b

When I remove the .with_middlewares call with the broker I am able to start the worker (and process tasks) with two workers. However, what is interesting is that when I reduce it to one worker I see the same output as one worker with prometheus middleware enabled: FileNotFoundError

Any ideas? I am relatively new to python so kind of scratching my head at what else could be useful to debug, please let me know if you need any additional info! Thanks in advance!

[nickdichev-firework]

@s3rius thanks for the fix, I was debugging the middleware a bit today and I noticed that the startup callback wasn't being executed.

I read the prometheus client documentation's section on multiprocessing and I noticed a couple of things. I implemented some changes which made the metrics start working with two workers. I still see the issue with one worker, not sure what that's about 🤷‍♂️

I opened #176 with my changes. Please take a look and let me know what you think! Again, I am not much of a python dev so lmk if anything looks off. Thanks again for the great work here, I learned a lot reading this codebase.

[s3rius]

@nickdichev-firework Did you update you taskiq?

[s3rius]

Yesterday I made changes. Today I published a new release. Please try version https://github.com/taskiq-python/taskiq/releases/tag/0.8.3.

The problem with registries is that it shows only taskiq's metrics. If user wants to add new metrics, they won't be shown by taskiq.

[nickdichev-firework]

@s3rius yes I still get the error on 0.8.3 -- the only way I've been able to resolve the Duplicated timeseries in CollectorRegistry: {'found_errors_total', 'found_errors', 'found_errors_created'} error is with my fork