Fix memory leak in `Supervisor` internal state

[durban]

Should fix issue #4490.

Reproducing that issue with the repro in the description there:

[info] Memory taken: 6 MB
[info] Memory taken: 520 MB

With this PR:

[info] Memory taken: 6 MB
[info] Memory taken: 7 MB

This issue also adds an (ugly) unittest, which inspects the internal state of the Supervisor. We could remove that, if we're satisfied with the fix, but I think it's useful to have it.

~~(Draft, because the test is racy, so I'm waiting for any possible CI failures.)~~