global map of widgets should utilize weak references to avoid memory leaks and excessive reload costs

[maxnoy]

Currently, all widgets created through the life of the kernel are referenced in a map Widgets.widgets. That map grows if the widgets are not closed down explicitly. It should be some version of WeakDictionary to avoid leaking memory and increasing the time to reinitialize the state of a notebook.

[maartenbreddels]

Good idea 👍 always wanted to raise this point, I hope it's this simple though.

[jasongrout]

This may not help as much as you think. If the comm is active, I think it will have a reference to the widget instance. If you close the widget (which shuts down the comm), it removes itself from the global cache.

[jasongrout]

I think we should send the entire state of all the widgets at once on page reload, instead of making tons of separate requests.

[maxnoy]

Is the memory leak on the kernel side a concern? It doesn't seem like the widgets get closed automatically.

[jasongrout]

Is the memory leak on the kernel side a concern? It doesn't seem like the widgets get closed automatically.

I think the point is that it isn't a memory leak if it's still communicating with something on the frontend. And the other point is that a weak reference to an active model won't do us any good because the comm still maintains a reference.

Another way to think about the lifecycle problem: the widget needs to be considered unreachable when both the frontend and the backend have no way of using it. That's a tricky problem.

[jasongrout]

This is sufficiently tricky that I'll push it to the Future milestone and put the discussion label on it.

[spott]

Is this still an issue? I believe I'm seeing this with another project, and I'm curious how I would go about explicitly closing down the widgets (do I do this from python, or from the javascript side?). Does anyone have a hint on where I should be doing the explicit closing?

[jasongrout]

We still don't use weak references. If anyone wants to explore it, please go for it.

[ianhi]

I'm curious how I would go about explicitly closing down the widgets (do I do this from python, or from the javascript side?). Does anyone have a hint on where I should be doing the explicit closing?

For future reference I think this should do it:

from ipywidgets import Widget
Widget.close_all()

[maartenbreddels]

Note that this problem is solved using https://github.com/widgetti/react-ipywidgets where we manage the widget lifecycle. e.g.:

If we now execute the next two cells:

We can see that all widgets that are managed by this react-ipywidgets render context are cleaned up.

Not only are widgets closed/destroyed when the full context is closed, but also when elements/widgets are not used anymore, i.e. during the lifetime of an application (conditional rendering etc)