haskell-language-server Run ghcide benchmark suite on HLS

Run ghcide benchmark suite on HLS

Open pepeiborra opened this issue 4 years ago • 6 comments

Ghcide has an excellent benchmark suite that tracks time and space usage across multiple revisions, experiments and examples. But by running it only on exe:ghcide, we are missing crucial data on the impact of HLS plugins on performance.

What are the steps required to run the ghcide benchmark suite on HLS plugins? And on HLS?

There' more than one option:

:heavy_check_mark: 1. Port the benchmark suite to HLS

We just move the Cabal stanzas (ghcide-bench and bench-hist) and the code across, add new experiments to exercise the relevant plugins, and run it on the whole of HLS + all the plugins.

2. Extract the benchmark suite to a package lsp-bench

As things stand right now, its' not even possible to package integration tests for plugins. All the integration tests live in HLS (func-test). We probably want to avoid ending up in the same situation with benchmarks.

A standalone package would be a way to allow each plugin to define its own benchmark suite.

Jan 12 '21 20:01 pepeiborra

I'm interested

Jan 17 '21 05:01 rayshih

Just started to read the code, it seems like we are using our own the benchmark utils. Should we consider to migrate to something like Criterion?

Jan 21 '21 07:01 rayshih

That seems unrelated to this issue. If you think migrating to Criterion would bring significant benefits, please feel free to open a new issue to discuss

Jan 22 '21 07:01 pepeiborra

It seems 1 is already done (actual hls bench came from ghcide one)