ipywidgets icon indicating copy to clipboard operation
ipywidgets copied to clipboard

Allow `Output.clear_output()` to be thread-safe.

Open eriknw opened this issue 4 years ago • 2 comments

Fixes #3260.

eriknw avatar Aug 29 '21 14:08 eriknw

Binder :point_left: Launch a binder notebook on branch eriknw/ipywidgets/threadsafe_clear_output

github-actions[bot] avatar Aug 29 '21 14:08 github-actions[bot]

There are some intricacies with this change that makes it not so straight-forward. First of all, I think a clear alternative to this change exists: Document that users can set the outputs trait to empty as an alternative way of clearing outputs, and explain in which situations the two ways of clearing the outputs makes the most sense.

Getting more into details, I note the following points:

  1. The content of an output widget can be modified in two ways: a. By manipulating the outputs trait, either directly, or via the convenience append_* methods on the output widgets. b. By entering its context manager, and using standard Jupyter protocol messages (e.g. via IPython.display.display() or IPython.display.clear_output() etc).
  2. I think that mixing the two patterns of modifying the outputs can lead to some race-conditions, even without threads. I qualify this with a "I think that" as I do not have any repros to demonstrate it.
  3. The context manager only works on the main thread.
  4. The current proposal in this PR would have the implementation of the clear_output method change from one way of modifying the outputs to another depending on the signature which it is called by, i.e. out.clear_output() differs from out.clear_output(wait=False). It would also change the default way used in most cases.
  5. The @out.capture(clear_output=True) decorator uses clear_output(), which would make it likely to mix the two patterns, increasing the risk of races.

In sum, I think this change would be likely to make the behavior more mysterious/confusing. Instead I think we should more clearly document the current pattern and suggestions. An example for of a documentation paragraph could be:

There are two different ways of changing the outputs of an Output widget:
- By manipulating the `outputs` trait of the widget. This includes calling the `Output.append_*` methods.
- By using it as a context manager (in a `with:` clause), and using standard methods for manipulating outputs, i.e. `IPython.display` etc.

Mixing the two ways is not recommended, as it is likely to cause race conditions, giving unpredictable behavior.

Note: For multi-threaded scenarios, only the first method is going to be effective, as the context manager requires that the code is running on the main thread.

We can then also add a note in the docstring of clear_output saying that it will not work from another thread than the main thread. Ideally we would also add a similar note for the context manager (__enter__ method? __init__? class docstring?).

vidartf avatar Nov 30 '21 18:11 vidartf