swift-kafka-client icon indicating copy to clipboard operation
swift-kafka-client copied to clipboard

Update Benchmark set up and persist benchmarks

Open mimischi opened this issue 1 year ago • 2 comments

We have recently updated our toolchain version to 5.9, but have never updated the benchmark suite to use 5.9 or newer. Also, it does not look like we ever commited a baseline to this repository.

While at it, wiring up GitHub actions to run the benchmarks for every pull request, as well as on a merge into main. Re-uses a lot of the heavy lifting that was defined in apple/swift-nio already.

Note that nightly-main appears broken at the moment. The Benchmark tool gets stuck, and initial debugging point to it never finishing _sendAndAcknowledgeMessages when producing a set of messages before every benchmark.

mimischi avatar Nov 25 '24 13:11 mimischi

Right. So the benchmark performs worse on GitHub Actions than on my local machine. What's the blessed approach of introducing a baseline then?

mimischi avatar Nov 25 '24 14:11 mimischi

Extract the values from GH actions and use those as the baseline, sadly.

Lukasa avatar Nov 25 '24 14:11 Lukasa