Speeding up the visualization of masks

[calebrob6]

show_anns(anns) in automatic_mask_generator_example.ipynb calls ax.imshow for each mask in the output which can be very slow for large images.

This PR changes show_anns to compute a single overlayed image that can be visualized with a single call to ax.imshow

[calebrob6]

Because the diff doesn't show inline, here's the new function for anyone who is interested:

def show_anns(anns):
    if len(anns) == 0:
        return
    sorted_anns = sorted(anns, key=(lambda x: x['area']), reverse=True)
    ax = plt.gca()
    ax.set_autoscale_on(False)

    img = np.ones((sorted_anns[0]['segmentation'].shape[0], sorted_anns[0]['segmentation'].shape[1], 4))
    img[:,:,3] = 0
    for ann in sorted_anns:
        m = ann['segmentation']
        color_mask = np.concatenate([np.random.random(3), [0.35]])
        img[m] = color_mask
    ax.imshow(img)

[kadirnar]

Because the diff doesn't show inline, here's the new function for anyone who is interested:

def show_anns(anns):
    if len(anns) == 0:
        return
    sorted_anns = sorted(anns, key=(lambda x: x['area']), reverse=True)
    ax = plt.gca()
    ax.set_autoscale_on(False)

    img = np.ones((sorted_anns[0]['segmentation'].shape[0], sorted_anns[0]['segmentation'].shape[1], 4))
    img[:,:,3] = 0
    for ann in sorted_anns:
        m = ann['segmentation']
        color_mask = np.concatenate([np.random.random(3), [0.35]])
        img[m] = color_mask
    ax.imshow(img)

What is the difference? Can you do it using opencv library?

How can I add? https://github.com/kadirnar/segment-anything-video/blob/main/metaseg/demo.py#L47-L72

[calebrob6]

@kadirnar I've pasted the original function below -- you can see that ax.imshow(...) is being called once for each mask (which is very slow). You could likely re-do this with OpenCV, but I don't see the point of doing that as this works fine.

def show_anns(anns):
    if len(anns) == 0:
        return
    sorted_anns = sorted(anns, key=(lambda x: x['area']), reverse=True)
    ax = plt.gca()
    ax.set_autoscale_on(False)
    polygons = []
    color = []
    for ann in sorted_anns:
        m = ann['segmentation']
        img = np.ones((m.shape[0], m.shape[1], 3))
        color_mask = np.random.random((1, 3)).tolist()[0]
        for i in range(3):
            img[:,:,i] = color_mask[i]
        ax.imshow(np.dstack((img, m*0.35)))

[csm-consight]

@calebrob6 Thanks!!!

[HannaMao]

@calebrob6 Thank you for the contribution! Could you please rebase the pr to the latest main? I could merge it afterwards.

[calebrob6]

@HannaMao done, thanks!

[Dammy-Hawey]

Good afternoon Please I need aid concerning my project, based on your past work I think I'm interested in your research skill