maplibre-gl-js icon indicating copy to clipboard operation
maplibre-gl-js copied to clipboard

Add descriptions to benchmarks

Open kircher1 opened this issue 3 years ago • 4 comments

The benchmarks provide a great framework for measuring performance deltas between main/release and a working branch.

However, I haven't found any code comments within the individual benchmarks describing what is being measured... Digging into the code to understand each benchmark is tedious.

Benchmarks should be cleaned up with additional class-level comments that explain what the benchmark is doing.

Better yet, it would be great if the description could be added into the benchmarking page itself, below each benchmark title.

image

kircher1 avatar Aug 23 '22 17:08 kircher1

I generally agree, but I think this issue should be part of #982 or at least addressed there. There's much to improve...

HarelM avatar Aug 23 '22 18:08 HarelM

Better documentation for the current benchmarks could also be addressed separately of redesigning them. It wasn't clear to me if in #982 you mean to change the individual benchmarks themselves, but if so, then I agree addressing the lack of docs would be good to do as part of that.

kircher1 avatar Aug 23 '22 18:08 kircher1

If we redesign, which I think would be a better use of anyone's time, it will create a situation where the work we did in order to document to be less relevant. I personally prefer to invest in redesign and document what the end state will be instead of using this time to reverse engineer the current code only in order to document it. But it's a free county :-)

HarelM avatar Aug 23 '22 18:08 HarelM

Any understanding and subsequent documentation of what a benchmark is doing will be helpful now or in a redesigned system-- that doesn't seem like wasted work to me unless you plan to toss the current benchmarks and start from scratch as part of #982.

kircher1 avatar Aug 23 '22 19:08 kircher1