`asyncio.exceptions.CancelledError` in CI

[jakirkham]

Didn't see this issue in past successful CI runs. Though it is showing up in current CI runs. Missed previously as CI didn't actually fail. Snippet of the traceback below (this repeats a few times).

Traceback (most recent call last):
  File "/home/vsts/work/1/s/distributed/.asv/env/34416fe3a6147c2fc95c423511ab564e/lib/python3.8/site-packages/distributed/utils.py", line 761, in wrapper
    return await func(*args, **kwargs)
  File "/home/vsts/work/1/s/distributed/.asv/env/34416fe3a6147c2fc95c423511ab564e/lib/python3.8/site-packages/distributed/client.py", line 1211, in _reconnect
    await self._ensure_connected(timeout=timeout)
  File "/home/vsts/work/1/s/distributed/.asv/env/34416fe3a6147c2fc95c423511ab564e/lib/python3.8/site-packages/distributed/client.py", line 1241, in _ensure_connected
    comm = await connect(
  File "/home/vsts/work/1/s/distributed/.asv/env/34416fe3a6147c2fc95c423511ab564e/lib/python3.8/site-packages/distributed/comm/core.py", line 313, in connect
    await asyncio.sleep(backoff)
  File "/home/vsts/work/1/s/distributed/.asv/env/34416fe3a6147c2fc95c423511ab564e/lib/python3.8/asyncio/tasks.py", line 659, in sleep
    return await future
asyncio.exceptions.CancelledError

It's unclear whether this is actually an issue with the benchmarks or if it is a Distributed bug. This it is related to the WorkerRestrictionsSuite benchmark, which caused some issues when fixing CI (due to upstream changes in Distributed). So there may be more issues still there.