Add a benchmark which uses Google benchmark library
~~Automatically generate performance benchmark information for website using Google Benchmark~~
Add a google benchmark based benchmark application
Codecov Report
Attention: Patch coverage is 0% with 108 lines in your changes missing coverage. Please review.
Project coverage is 83.15%. Comparing base (
573effc) to head (e8efae5). Report is 1 commits behind head on dev.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| ...amples/PcapPlusPlus-benchmark/benchmark-google.cpp | 0.00% | 108 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## dev #1392 +/- ##
==========================================
- Coverage 83.35% 83.15% -0.20%
==========================================
Files 276 277 +1
Lines 48086 48196 +110
Branches 9721 9697 -24
==========================================
- Hits 40080 40077 -3
- Misses 7115 7223 +108
- Partials 891 896 +5
| Flag | Coverage Δ | |
|---|---|---|
| fedora40 | 75.19% <0.00%> (-0.22%) |
:arrow_down: |
| macos-13 | 80.65% <0.00%> (-0.21%) |
:arrow_down: |
| macos-14 | 80.65% <0.00%> (-0.21%) |
:arrow_down: |
| macos-15 | 80.62% <0.00%> (-0.22%) |
:arrow_down: |
| mingw32 | 70.87% <ø> (-0.05%) |
:arrow_down: |
| mingw64 | 70.88% <ø> (-0.01%) |
:arrow_down: |
| npcap | 85.22% <ø> (-0.09%) |
:arrow_down: |
| rhel94 | 75.05% <0.00%> (-0.22%) |
:arrow_down: |
| ubuntu2004 | 58.60% <0.00%> (-0.15%) |
:arrow_down: |
| ubuntu2004-zstd | 58.72% <0.00%> (-0.17%) |
:arrow_down: |
| ubuntu2204 | 74.96% <0.00%> (-0.21%) |
:arrow_down: |
| ubuntu2204-icpx | 61.41% <ø> (+<0.01%) |
:arrow_up: |
| ubuntu2404 | 75.22% <0.00%> (-0.23%) |
:arrow_down: |
| unittest | 83.15% <0.00%> (-0.20%) |
:arrow_down: |
| windows-2019 | 85.33% <ø> (-0.01%) |
:arrow_down: |
| windows-2022 | 85.35% <ø> (-0.02%) |
:arrow_down: |
| winpcap | 85.32% <ø> (ø) |
|
| xdp | 50.50% <0.00%> (-0.15%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Where can I see the benchmark result?
@tigercosmos Check "Run benchmark" step https://github.com/seladb/PcapPlusPlus/actions/runs/10644856460/job/29510043289
Approx. 10Gbps parsing, 1Gbps crafting performance in GitHub CI (just looking bytes per sec). Also items per sec is a good indicator which can we tabulate as for mean packet sizes (For example, if avg packet length 64, 512, 1024, 1500 etc -> size* items per sec *8
Will it show a warning if the benchmark performance decreases too much? Otherwise, people will not click into the CI and check the result.
@tigercosmos Actually my aim is to generate and update the benchmark results automatically. I enabled PR runs for just testing. I'll disable it when the work is done, and the runs will be only from master or dev. Because benchmark results can be very flaky without a dedicated machine/runner unfortunately. This workflow and integration with Google benchmark are just for easier benchmarking in local PCs and to give an idea about to performance in PcapPlusPlus website (since current benchmark really old)
@egecetin I'm wondering if there's much value in publishing our benchmark without comparing PcapPlusPlus to other libraries. What do you think?
@seladb Actually it is just for give an idea about the performance in terms of packet per second and for easier local benchmarking (both disk and cpu sides). Github's hardware is good enough to get good results. This is why i added long sentence to README and a workflow to CI. I agree it alone does not have true meaning unfortunately so why I left the old benchmark also. I can (at least might try 🙂) also update to comparing with other libraries periodically. Maybe once a month or at least manually run after every release in a different repo.
Maybe what we really need is to add this as another example (and move the instructions to the internal README)? It's probably not worth running it in CI and uploading the results to the web-site repo
@seladb It is ready to review if you have time