dramatiq icon indicating copy to clipboard operation
dramatiq copied to clipboard

Actor arguments do not get freed from memory

Open akshhack opened this issue 1 year ago • 1 comments

Hey @Bogdanp!

We've been using dramatiq for a while, however, have started noticing issues in memory leaks as the size of the actor payloads have increased. We are passing in pretty large dicts and lists (w/ PickleEndoer) and have noticed that each time the actor is run, it leaves the arguments on the heap.

Running actors multiple times leaves the arguments on the heap repeatedly and this grows and OOMs out our pods in K8s. Is this something you're aware of or see how can be a potential issue?

We don't have workarounds; we'd love to continue using dramatiq but are pretty stumped on this.

akshhack avatar Jul 05 '23 15:07 akshhack

Hard to help here without a minimal example. It seems unlikely that Dramatiq itself is retaining this data, so I'd look at any custom middleware you might have or if any of the de-pickled data does something funky on deserialize.

Bogdanp avatar Jul 08 '23 07:07 Bogdanp