Memory leak when using ImageObject in a loop or hanging process

[stenson]

I've been using drawBot as a module in a hanging process (so my script runs immediately on save), and have been using ImageObjects, which works fine, but I noticed the memory consumption of the hanging process increases linearly every time I use an ImageObject. If the image is 1000x1000 pixels, the memory jump is about ~20 MB every time the script is run, or about ~40 MB for a 2000x2000 image, meaning the memory consumption of the process can quickly become quite large.

Quitting and restarting the process fixes the issue completely, but I'm wondering if there's any way to remedy the situation? I've poked around in the imageObject.py file looking for any culprits, but so far haven't come up with anything, though my best guess would be that some Cocoa-level image representation is managing to hang around after its lifecycle.

Anyway obviously not a pressing issue & would love to be able to help out in fixing it — just haven't had any breakthroughs on my own, so I wanted to see if anyone working on the project had an inklings as to a fix (or an explanation of something I’m doing wrong!).

Reproducing

The DrawBot app itself seems to handle the memory better, but with code like the below, I can get the app to show similar memory usage while it's running (though the app does seem like it's able to clean up after itself once the script has finished and the displayed PDFs are wiped).

from time import sleep

def render():
    newPage(1000, 1000)
    if True:
        im = ImageObject()
        with im:
            size(1000, 1000)
            fontSize(500)
            text("Blur", (0, 0))

        im.gaussianBlur(radius=10)
        im.pixellate()
        image(im, (0, 0))

for x in range(0, 25):
    render()
    sleep(1)

(The sleep is just to make it easier to watch the memory consumption in the Activity Monitor)

Here’s some alternate code for the drawBot-as-a-module use-case — not a hanging process as described above but has sleeps in there again so it mimics that kind of behavior. (Requires an installation of psutil)

import os
import psutil
from drawBot import *
from time import sleep

process = psutil.Process(os.getpid())

def mb(bytes):
    r = float(bytes)
    for i in range(2):
        r = r / 1024
    return(r)

def render():
    for x in range(0, 3):
        newDrawing()
        size(1000, 1000)

        if True:
            im = ImageObject()
            with im:
                size(1000, 1000)
                fontSize(500)
                text("Blur", (0, 0))

            #im.gaussianBlur(radius=10)
            #im.pixellate()
            image(im, (0, 0))

        endDrawing()

        print(f"> rendered {x} ({os.getpid()})")
        print("> memory", mb(process.memory_info().rss))
        sleep(1)
    
if __name__ == "__main__":
    render()

[justvanrossum]

This may be the result of "autoreleased" objects, a Cocoa/Objective-C concept where objects will be released the next time through the event loop. That is obviously a problem if there's no event loop.

It's often possible to workaround this using an explicit autorelease pool like this:

        pool = AppKit.NSAutoreleasePool.alloc().init()
        try:
            ... # use code that creates autoreleased objects
        finally:
            del pool

The DrawBot source code uses this pattern here and there.

If this happens to make a difference in your case, we could try to pinpoint the code in DB that causes it, and incorporate the workaround there.

[stenson]

Definitely helps quite a bit! Looks like the memory added per ImageObject used is reduced by ~5x w/ that added — definitely still some memory leaking each time the ImageObject is used (w/ no ImageObject, the memory added each time is only ~0.02 MB), but that's a big help — thanks!

[dpmellis]

I had the exact same issue with ImageObject, and the autorelease pool fixed it, thanks!