Fix broken pipe during test and gracefully exit the server

[soulomoon]

gracefully exit the server

[soulomoon]

Key idea:

1. Wait for the refactor loop to finish in shutdown handler.
2. After sending the shutdown response, stop sending further response to client. (This is done in lsp)

[soulomoon]

What's been done

1. Refactor exit logic

   • Previously, shutdown and Shake database cleanup only happened in the shutdown handler.
   • Now, the cleanup is moved into the finally clause of the reactor thread, ensuring it always runs on reactor exit.
   • Also adapted to a modified LSP package branch so the server can now gracefully exit. https://github.com/haskell/lsp/pull/622#issue-3349324637
   • Workaround the case that workthread might not exit because the threadcancel exception is swollen by downstream code.

2. Improve logging

   • Added clearer and more structured logging to aid debugging.

3. Add flakiness testing support

   • Introduced a flakiness testing script and CI integration.
   • Makes it easier to detect and diagnose flaky behavior in development and CI runs.

[fendor]

@soulomoon Is this ready for review?

[soulomoon]

@soulomoon Is this ready for review?

yes, it is. thanks.

[soulomoon]

odd 9.10 windows failing for Access violation in generated code when reading 0x7ff4933977a0

[fendor]

@soulomoon Probably just needs to rebased on top of #4768