StubBroker problems with Prometheus middleware.

[jenstroeger]

Issues[^discussion]

I’m trying to use the StubBroker in unit tests as described in this section. I noticed one problem and wanted to raise it for discussion: Using the default middlewares

https://github.com/Bogdanp/dramatiq/blob/a104a0bf999e6eb655c8d21d9d464e6697c7088b/dramatiq/middleware/init.py#L44-L47

for the StubBroker includes the Prometheus middleware which, when run in a pytest context, leaves the following errors after testing:

Exception ignored in: <_io.FileIO name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/gauge_livesum_25420.db' mode='ab+' closefd=True>
ResourceWarning: unclosed file <_io.BufferedRandom name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/gauge_livesum_25420.db'>
Exception ignored in: <_io.FileIO name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/histogram_25420.db' mode='ab+' closefd=True>
ResourceWarning: unclosed file <_io.BufferedRandom name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/histogram_25420.db'>
Exception ignored in: <_io.FileIO name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/counter_25420.db' mode='ab+' closefd=True>
ResourceWarning: unclosed file <_io.BufferedRandom name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/counter_25420.db'>

I’ve not yet dug into this issue, but could probably take a look.

For the time being I work around this by removing the Prometheus middleware from the StubBroker[^arg] as we discussed here:

_, *middleware = default_middleware
broker = StubBroker(middleware=[m() for m in middleware])

I’m unsure if that’s related to the mentioned Prometheus Gotchas.

What OS are you using?

macOS Ventura 13.2

What version of Dramatiq are you using?

>>> dramatiq.__version__
'1.14.0'

What did you do?

See above.

What did you expect would happen?

No warnings from the Prometheus middleware.

What happened?

Warnings from the Prometheus middleware.

[^discussion]: I think the template needs updating as it still references the retired Reddit instead of the Groups.io link.

[^arg]: It might be a good idea to document the middleware arg for the StubBroker here?

[jenstroeger]

Note that the Prometheus middleware relies on various middleware hooks to set up and tear down — make sure that those hooks execute in the pytest context!