ginkgo
ginkgo copied to clipboard
Add some micro-benchmarking support to Ginkgo
This PR adds some micro-benchmarking support to benchmark all the small functionalities of Ginkgo. For example, you can get information like the following:
-
Executor creation and run timings and overheads.
-
Array creation, copy, move timings and differences.
-
More specific backend-specific timings as well are possible, example for arrays with CUDA:
Of course there are a few things that we need to discuss:
- Timings for GPUs is from the host side, but that may be okay.
- The repetitions for the function calls is not something that can be clearly controlled through google benchmark, which as discussed before is a downside of this.
I think this type of micro-benchmarking information can be really useful in detecting performance degradations and can atleast give us a hint if some commit/change regressed the performance.
Kudos, SonarCloud Quality Gate passed!
0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells
No Coverage information
9.5% Duplication
Codecov Report
Merging #790 (ff9b336) into develop (6127408) will decrease coverage by
0.00%
. The diff coverage isn/a
.
@@ Coverage Diff @@
## develop #790 +/- ##
===========================================
- Coverage 94.74% 94.73% -0.01%
===========================================
Files 434 434
Lines 35705 35706 +1
===========================================
Hits 33827 33827
- Misses 1878 1879 +1
Impacted Files | Coverage Δ | |
---|---|---|
core/base/extended_float.hpp | 91.26% <0.00%> (-0.98%) |
:arrow_down: |
omp/reorder/rcm_kernels.cpp | 97.53% <0.00%> (-0.61%) |
:arrow_down: |
include/ginkgo/core/base/range_accessors.hpp | 78.94% <0.00%> (+5.26%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 6127408...ff9b336. Read the comment docs.
@upsj, Thank you for the review and the comments. I will try to update it soon.
Thanks for this, I think using google benchmark for these parts can make sense, since that's pretty much what it was designed for. I have a few remarks after a quick glance at the code, although I didn't look into it in depth:
- I like the name
micro_benchmarks
for the directory as that explains what should go in there. - I added some overhead benchmarks to our base benchmark line, maybe that can go in there instead (or in addition?).
- We maybe should use the full name
googlebenchmark
for the targets and some other aspects, at first it wasn't clear to me whatGBench
andmbench
were. I would stick to usingmicro_benchmarks
always for the Ginkgo side, and the fullgooglebenchmark
or whichever is the original name for the third party. - A big question is CUDA: by default the timer is CPU only and that's terrible for doing GPU micro-benchmarking. I know it's possible to add custom timers but I'm not sure how extensible they are, back when I had tried it (many years ago) that aspect wasn't satisfactory, maybe it's better now?
@tcojean, thanks. I will look into GPU timers and see if it extensible from within googlebenchmark. I agree with your other suggestions.
Error: The following files need to be formatted:
core/micro_bench/base/array.cpp
core/micro_bench/base/executor.cpp
cuda/micro_bench/base/array.cu
cuda/micro_bench/base/executor.cu
You can find a formatting patch under Artifacts here or run format!
if you have write access to Ginkgo
Kudos, SonarCloud Quality Gate passed!
0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells
No Coverage information
9.5% Duplication