mo.mpl.interactive() leaves blank figures if you're not looking

[ggggggggg]

Describe the bug

Often when I run, or re-run, a notebook containing calls to mo.mpl.interactive() while being in a separate tab or having my browser minimized (I'm using chrome) I come back to blank plots. Then on each cell I have to run it again to actually see the plot.

Environment

{ "marimo": "0.8.0", "OS": "Windows", "OS Version": "10", "Processor": "Intel64 Family 6 Model 85 Stepping 4, GenuineIntel", "Python Version": "3.10.11", "Binaries": { "Browser": "127.0.6533.120", "Node": "--" }, "Requirements": { "click": "8.1.7", "importlib-resources": "missing", "jedi": "0.19.0", "markdown": "3.6", "pymdown-extensions": "10.8.1", "pygments": "2.16.1", "tomlkit": "0.12.5", "uvicorn": "0.30.1", "starlette": "0.37.2", "websockets": "12.0", "typing-extensions": "4.7.1", "ruff": "0.5.5" } }

Code to reproduce

I reproduce with this notebook, tested 3 variation, all of which start from the marimo workspace browser. Between variations I close the previous tab and click the red shutdown button in the workspace browser.

1. leave the tab active and watch, all plots appear
2. click to another tab and wait ~20 seconds, then click back, all plots blank
3. minimize browser, look at outlook for 20 seconds, then click back, all plots blank

import marimo

__generated_with = "0.8.0"
app = marimo.App(width="medium")


@app.cell
def __():
    import marimo as mo
    import pylab as plt
    def show():
        return mo.mpl.interactive(plt.gcf())
    return mo, plt, show


@app.cell
def __(plt, show):
    plt.plot(range(100))
    show()
    return


@app.cell
def __(plt, show):
    plt.plot(range(100))
    show()
    return


@app.cell
def __(plt, show):
    plt.plot(range(100))
    show()
    return


@app.cell
def __(plt, show):
    plt.plot(range(100))
    show()
    return


@app.cell
def __(plt, show):
    plt.plot(range(100))
    show()
    return


if __name__ == "__main__":
    app.run()

[mscolnick]

I tried playing around with this and could not reproduce it. Are you able to reproduce it consistently?

[ggggggggg]

I can replicate 100% of the time or close to it.

[mutongx]

I also have the similar experience when using mo.mpl.interactive. The browser console says websocket connection is closed when it happens. But I haven't dig into it.

marimo backend is running in Ubuntu 22.04, and my browser is Microsoft Edge running in Windows 10.

[mscolnick]

@ggggggggg or @mutongx, do you see the same in Jupyter?

[ggggggggg]

I don't use jupyter very often, and I never use interactive plots in jupyter. I was never able to get interactive matplotlib plots to work reliably on any OS in jupyter. If you can provide an example notebook that uses the same backend or whatever should be similar I'm happy to try it.

[mscolnick]

sorry I dont use jupyter either. I was just trying to bisect if it is a marimo issue, matplotlib issue or browser issue.

[mutongx]

It looks like that ipyml does not use WebAgg, instead it uses Jupyter's own widget system, which uses their own communication mechanism. WebAgg uses vanilla WebSocket instead. I think it is not useful to compare WebAgg with ipympl.

Strangely, I was trying to replicate the problem just now and the "websocket closed" problem disappeared. Instead, I get a blank figure if the browser is minimized during a cell run:

Then I use the pan tool to drag the figure, I get the following, notice that the figure's border disappeared:

So I think the current problem may be related to some canvas redraw behavior? Like the WebAgg wants to save computation so it doesn't redraw the border.

---

Some extra information about the websocket problem:

I remember that the iframe size in the frontend is small as shown in the following image when I see websocket is closed in the devtools console (I emulated this by blocking port 10000):

@ggggggggg does your iframe look like this, or it is in the correct size? It may help to identify the actual problem.

[mscolnick]

@mutongx - i can get in the same state as your. my hunch is it is a performance thing (ignore events when the browser is not focused), either by matplotlib or the browser itself.

[ggggggggg]

I don't recall seeing a smaller figure. I don't know how to block websockets, so I'd need more guidance to try it myself.

[mutongx]

@ggggggggg Sorry but I don't know how to block a port in Windows either.

You can try to use the pan tool to see whether the figure is draggable. If it is responsive, then the websocket connection is alive, and I think it will be a upstream problem. If not, you can check your browser's DevTools console for error logs to see if it is a websocket connection problem (right click at the page's blank area, click "Inspect", then look for error logs in the "Console" tab). The error log should look like this:

[mutongx]

@mscolnick yeah, but I definitely remember that I've encountered websocket problem quite frequently several days ago. I suppose it is related to unstable network or something, as I'm using marimo in a remote machine through VS Code's port forwarding.

I think marimo can at least implement a websocket reconnect for improved stability. However, I do recognize that the current mo.mpl.interactive() implementation is not polished, like using an iframe, require a new port, hardcoded to localhost, so I'm OK with it if marimo needs more time to fix it (and I will be happy to contribute if I figured out how to do it in a "good" way, but I just don't have spare time recently).

[mscolnick]

@mutongx - I agree it does need to be polished, thank you for noting the areas it could be improved.

if you have time and can contribute, that would be amazing. if not, i can try to get to it next week.

[ggggggggg]

When I come back to a blank window, it is responsive. But it is quite odd. I tried using the zoom tool, and find that the outer area of the plot doesn't change the cursor on the zoom tool and can't be used to zoom, but the inner area can. When I use the zoom tool on the inner area, then hit home I get the following.

Hitting home before using the zoom tool gets no response.

Compared to this if I re-run the cell: