Use weakref to resolve kernel side memory leaks

[fleming79]

This PR fixes kernel side memory leaks and makes some performance tweaks. The primary user facing change is that invalid/closed children in boxes are quietly dropped.

Features:

• [x] Fixes https://github.com/jupyter-widgets/ipywidgets/issues/1345.
• [x] Strong references between widgets and comms replaced with a Ref.
• [x] _instance dict replaced with a weak value dict.
• [x] The comm should now more reliably auto magically instantiate.
• [x] Widgets will automatically close once the main reference is lost.
• [x] The decorator _show_traceback is replaced with a method _show_traceback.
• [x] Special Children class to validate items and closed widgets (faster), quietly discarding invalid children.
• [x] Option for widgets to removed from children of box dynamically.
• [x] Fixed loading state via jupyter.widget.control comm channel.
• [x] Tests for widget garbage collection.

Examples

Garbage collection

In a fresh notebook running this code.

import ipywidgets as ipw
import asyncio

b = ipw.Button(description='Test')
assert any(ipw.widget._instances)
display(b)
await asyncio.sleep(2)
del b
assert not any(ipw.widget._instances)

Before

The assertion fails due to strong references

• widget._instances -> mapping of id to instance
• A callback between the widget method _handle_msg and its comm callback registry.

After

Before sleeping.

After sleeping.

Note the assertion is good, meaning the widget has been deleted.

Dropping invalid widgets

The new Children class replaces TypedTuple providing an optimised (faster) trait specific to validating a tuple of Widgets. It will also convert any type of iterable such as sets and generators to a tuple. The change in behaviour means that Box.children drops invalid elements rather than raising an error.

import ipywidgets as ipw

closed_button = ipw.Button(description="test")
closed_button.close()
b = ipw.Box(children=["Not a widget", ipw.Button(description="test"), closed_button])
display(b)
assert len(b.children) == 1

Before

After

Passing children that aren't widgets, or widgets that are closed are simply discarded rather than raising an error.

Notes

JSLink is a special case, which may need further consideration.

[github-actions[bot]]

:point_left: Launch a binder notebook on branch fleming79/ipywidgets/main