maplibre-gl-js

maplibre-gl-js copied to clipboard

We should either remove bench/gl-stats.js or figure out what it is good for and use it possibly in CI.

https://github.com/maplibre/maplibre-gl-js/blob/f87090b70b8a6fbde1a779c44933ce32bffc5d2c/.circleci/config.yml#L188-L199

I fixed the script itself in #765. It was originally introduced by https://github.com/mapbox/mapbox-gl-js/pull/8645. I don't know a lot about web gl but they do look useful. They are fast enough to generate and bisect a possible regression in reasonable time though so as of now I don't see the need to generate them in ci.

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 30 days.

Where are we on this? What are our options in terms of CI? Maybe we can have a manual CI that can run this and publish the results somehow?

@wipfli where are we with this? Should we talk about it in the next steering committee?

Yes, that's a good idea. Maybe we can run the benchmarks on dedicated hardware as part of the MapLibre GL JS CI...

Steering committee discussion:

1. This should be part of a CI run
2. We should only run the current version and not compare to other versions to reduce run time
3. We should try and use a dedicated hardware to run this so the results would be similar as much as possible
4. We should try and define the "important"/"sanity" runs to run in the CI to reduce time

My personal note - we need a better way to determine if the benchmark as passing or not, I personally and baffled by the results. And at a minimum we should run transpilation on the code as part of the CI to make sure it doesn't break (if we are not doing this already as part of the jest typescript checks).