dramatiq
dramatiq copied to clipboard
Actor arguments do not get freed from memory
Hey @Bogdanp!
We've been using dramatiq for a while, however, have started noticing issues in memory leaks as the size of the actor payloads have increased. We are passing in pretty large dicts and lists (w/ PickleEndoer) and have noticed that each time the actor is run, it leaves the arguments on the heap.
Running actors multiple times leaves the arguments on the heap repeatedly and this grows and OOMs out our pods in K8s. Is this something you're aware of or see how can be a potential issue?
We don't have workarounds; we'd love to continue using dramatiq but are pretty stumped on this.
Hard to help here without a minimal example. It seems unlikely that Dramatiq itself is retaining this data, so I'd look at any custom middleware you might have or if any of the de-pickled data does something funky on deserialize.