Flaky `CancelCompileLspSuite`

[tanishiking]

Describe the bug

These days, I see tests.CancelCompileLspSuite fails randomly.

[error] ==> X tests.CancelCompileLspSuite.basic  4.01s java.lang.AssertionError: assertion failed: expected didOpen future to fail with Cancellation Exception. If this happens frequently for unrelated changes, then this may be a flaky test that needs refactoring. If this assertion is flaky, feel free to remove it until it's refactored.
[error]     at scala.Predef$.assert(Predef.scala:279)
[error]     at tests.CancelCompileLspSuite.$anonfun$new$9(CancelCompileLspSuite.scala:44)
[error]     at tests.CancelCompileLspSuite.$anonfun$new$9$adapted(CancelCompileLspSuite.scala:36)
[error]     at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
[error]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[error]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error]     at java.lang.Thread.run(Thread.java:833)

This is not much frustrating as it usually recovers by re-running the tests, but it would be awesome if we can stabilize the test :)

Expected behavior

Stable test result for CancelCompileLspSuite.

Operating system

Any operating system on GitHub actions.

Editor/Extension

No response

Version of Metals

v0.11.2

Extra context or search terms

flaky test / cancel compile

[tgodzik]

Might actually be related to https://github.com/scalameta/metals/issues/3754