opentelemetry-profiling
opentelemetry-profiling copied to clipboard
Initial benchmark setup
This PR moves benchmarking code to opentelemetry repo. It is based on the code from this repo.
I made some additional changes and improvements:
- added a pprof parser, so now you can provide profiles in
collapsedandpprofformats - made it so that anonymizer only runs on files marked as
sensitive. Unfortunately anonymization affects compression ratio, so I figured we might want to use anonymizer as little as possible. - added multiple encoders:
json,pprof,pprof-gzip, to showcase that we can compare different encoder implementations. Interesting next step would be to add an otel events encoder (perhaps with gzip compression) and compare it to pprof implementation. - added a few new profile examples,
goappandgoapp.sensitive, to showcase the difference in encoding data before and after anonymization - implemented collection of byte size and CPU time (duration) metrics
- added some basic formatting for the benchmarking results when printed in the terminal:
Click to expand terminal example
Filename: basic.collapsed.intermediary
+------------+----------+---------+-----------+---------+
| ENCODER | DURATION | % DIFF | BYTE SIZE | % DIFF |
+------------+----------+---------+-----------+---------+
| pprof-gzip | 251.7µs | | 114 bytes | |
| pprof | 3.5µs | -98.59% | 89 bytes | -21.93% |
| json | 469ns | -99.81% | 49 bytes | -57.02% |
+------------+----------+---------+-----------+---------+
Filename: basic.sensitive.collapsed.intermediary
+------------+----------+---------+-----------+---------+
| ENCODER | DURATION | % DIFF | BYTE SIZE | % DIFF |
+------------+----------+---------+-----------+---------+
| pprof-gzip | 151.5µs | | 114 bytes | |
| pprof | 3µs | -98.05% | 89 bytes | -21.93% |
| json | 484ns | -99.68% | 49 bytes | -57.02% |
+------------+----------+---------+-----------+---------+
Filename: goapp.pprof.intermediary
+------------+----------+---------+------------+----------+
| ENCODER | DURATION | % DIFF | BYTE SIZE | % DIFF |
+------------+----------+---------+------------+----------+
| pprof-gzip | 2.6ms | | 16.60 KiB | |
| pprof | 982.1µs | -61.63% | 40.01 KiB | +141.06% |
| json | 438.3µs | -82.88% | 147.56 KiB | +788.95% |
+------------+----------+---------+------------+----------+
Filename: goapp.sensitive.pprof.intermediary
+------------+----------+---------+------------+----------+
| ENCODER | DURATION | % DIFF | BYTE SIZE | % DIFF |
+------------+----------+---------+------------+----------+
| pprof-gzip | 3.2ms | | 25.31 KiB | |
| pprof | 1.4ms | -56.08% | 40.01 KiB | +58.07% |
| json | 418.4µs | -86.94% | 147.56 KiB | +482.92% |
+------------+----------+---------+------------+----------+
- implemented a markdown rendered as well. For now this helps share results on Github. Maybe one day we'll make it post this on github PRs automatically:
Filename: basic.collapsed.intermediary
| Encoder | Duration | % diff | Byte Size | % diff |
|---|---|---|---|---|
| pprof-gzip | 251.7µs | 114 bytes | ||
| pprof | 3.5µs | -98.59% | 89 bytes | -21.93% |
| json | 469ns | -99.81% | 49 bytes | -57.02% |
Filename: basic.sensitive.collapsed.intermediary
| Encoder | Duration | % diff | Byte Size | % diff |
|---|---|---|---|---|
| pprof-gzip | 151.5µs | 114 bytes | ||
| pprof | 3µs | -98.05% | 89 bytes | -21.93% |
| json | 484ns | -99.68% | 49 bytes | -57.02% |
Filename: goapp.pprof.intermediary
| Encoder | Duration | % diff | Byte Size | % diff |
|---|---|---|---|---|
| pprof-gzip | 2.6ms | 16.60 KiB | ||
| pprof | 982.1µs | -61.63% | 40.01 KiB | +141.06% |
| json | 438.3µs | -82.88% | 147.56 KiB | +788.95% |
Filename: goapp.sensitive.pprof.intermediary
| Encoder | Duration | % diff | Byte Size | % diff |
|---|---|---|---|---|
| pprof-gzip | 3.2ms | 25.31 KiB | ||
| pprof | 1.4ms | -56.08% | 40.01 KiB | +58.07% |
| json | 418.4µs | -86.94% | 147.56 KiB | +482.92% |