libdatadog icon indicating copy to clipboard operation
libdatadog copied to clipboard

fix(crashtracker) Cache virtual memory address to speed up normalization

Open gleocadie opened this issue 7 months ago • 3 comments

What does this PR do?

This PR creates a normalizer which caches VMA (Virtual Memory Addresses) to speed up normalization.

Motivation

This PR addresses the issue reported in https://datadoghq.atlassian.net/browse/PROF-11744. Crashtracker does not produce crash reports since https://github.com/DataDog/libdatadog/pull/991/files (at least for Ruby)

Additional Notes

Added also build_id caching. build ids are important thingy for remote symbolication (in datadog backend)

How to test the change?

Describe here in detail how the change can be validated.

gleocadie avatar May 02 '25 07:05 gleocadie

Benchmarks

Comparison

Benchmark execution time: 2025-05-02 07:49:37

Comparing candidate commit 02d3083e in PR branch gleocadie/fix-normalization-regression with baseline commit fed767f7 in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 50 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/378282246310005

  • 🟥 execution_time [+9.551µs; +9.690µs] or [+14.063%; +14.268%]
  • 🟥 throughput [-1840920.797op/s; -1813520.845op/s] or [-12.502%; -12.316%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.972µs 5.040µs ± 0.034µs 5.042µs ± 0.027µs 5.062µs 5.093µs 5.096µs 5.100µs 1.14% -0.116 -1.062 0.67% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.035µs; 5.045µs] or [-0.093%; +0.093%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 72.345ms 72.605ms ± 0.223ms 72.537ms ± 0.081ms 72.680ms 72.982ms 73.451ms 74.252ms 2.36% 3.058 15.509 0.31% 0.016ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [72.574ms; 72.636ms] or [-0.043%; +0.043%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.235µs 3.223µs ± 1.458µs 3.018µs ± 0.036µs 3.054µs 3.677µs 14.090µs 15.134µs 401.51% 7.401 55.733 45.13% 0.103µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.021µs; 3.425µs] or [-6.270%; +6.270%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 5.987ms 5.999ms ± 0.008ms 5.998ms ± 0.004ms 6.002ms 6.009ms 6.030ms 6.069ms 1.18% 4.157 27.615 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [5.998ms; 6.000ms] or [-0.020%; +0.020%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 248.059ns 260.410ns ± 13.468ns 253.841ns ± 3.699ns 266.637ns 288.497ns 301.908ns 305.053ns 20.18% 1.473 1.334 5.16% 0.952ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [258.544ns; 262.277ns] or [-0.717%; +0.717%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 218.381µs 218.931µs ± 0.282µs 218.943µs ± 0.208µs 219.110µs 219.382µs 219.600µs 219.738µs 0.36% 0.269 -0.423 0.13% 0.020µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4550875.406op/s 4567653.498op/s ± 5887.977op/s 4567400.013op/s ± 4352.202op/s 4572223.880op/s 4576587.136op/s 4577903.387op/s 4579145.759op/s 0.26% -0.264 -0.430 0.13% 416.343op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 17.529µs 17.597µs ± 0.031µs 17.595µs ± 0.018µs 17.612µs 17.649µs 17.699µs 17.713µs 0.67% 0.743 1.074 0.18% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 56456299.740op/s 56828058.147op/s ± 100749.477op/s 56834816.863op/s ± 58814.757op/s 56893475.216op/s 56973887.504op/s 56993007.309op/s 57049933.554op/s 0.38% -0.730 1.040 0.18% 7124.064op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.261µs 10.370µs ± 0.104µs 10.312µs ± 0.028µs 10.489µs 10.552µs 10.566µs 10.572µs 2.52% 0.774 -1.161 1.00% 0.007µs 1 200
normalization/normalize_name/normalize_name/good throughput 94586066.823op/s 96437173.511op/s ± 960825.647op/s 96971563.292op/s ± 259323.513op/s 97165097.050op/s 97362986.772op/s 97430870.705op/s 97456605.772op/s 0.50% -0.766 -1.172 0.99% 67940.633op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [218.892µs; 218.970µs] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4566837.481op/s; 4568469.515op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [17.593µs; 17.601µs] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [56814095.238op/s; 56842021.055op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.356µs; 10.385µs] or [-0.139%; +0.139%] None None None
normalization/normalize_name/normalize_name/good throughput [96304012.317op/s; 96570334.705op/s] or [-0.138%; +0.138%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.332µs 2.386µs ± 0.020µs 2.385µs ± 0.009µs 2.392µs 2.431µs 2.438µs 2.447µs 2.59% 0.324 1.706 0.83% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.383µs; 2.389µs] or [-0.115%; +0.115%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 66.570µs 66.784µs ± 0.218µs 66.753µs ± 0.083µs 66.846µs 66.973µs 67.328µs 69.122µs 3.55% 6.930 67.591 0.32% 0.015µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [66.754µs; 66.814µs] or [-0.045%; +0.045%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.895µs 3.911µs ± 0.003µs 3.910µs ± 0.002µs 3.912µs 3.916µs 3.917µs 3.918µs 0.18% -0.400 2.894 0.08% 0.000µs 1 200
credit_card/is_card_number/ throughput 255256584.258op/s 255721716.514op/s ± 193185.969op/s 255724420.650op/s ± 126263.704op/s 255852878.395op/s 255996944.850op/s 256071058.395op/s 256744769.402op/s 0.40% 0.410 2.943 0.08% 13660.311op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 76.707µs 77.550µs ± 0.494µs 77.475µs ± 0.287µs 77.821µs 78.274µs 78.383µs 81.379µs 5.04% 2.790 17.961 0.64% 0.035µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12288233.959op/s 12895491.932op/s ± 80724.026op/s 12907468.269op/s ± 47695.567op/s 12949797.151op/s 12993859.588op/s 13010148.614op/s 13036639.627op/s 1.00% -2.569 15.848 0.62% 5708.051op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 70.878µs 71.423µs ± 0.430µs 71.356µs ± 0.197µs 71.569µs 71.994µs 72.345µs 75.870µs 6.33% 5.738 55.248 0.60% 0.030µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13180357.678op/s 14001635.762op/s ± 81675.465op/s 14014140.359op/s ± 38761.080op/s 14044741.319op/s 14079486.963op/s 14098586.284op/s 14108671.189op/s 0.67% -5.317 49.397 0.58% 5775.328op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.892µs 3.911µs ± 0.003µs 3.910µs ± 0.002µs 3.913µs 3.916µs 3.919µs 3.920µs 0.25% -0.485 4.628 0.08% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255097105.436op/s 255711853.388op/s ± 215527.448op/s 255747405.549op/s ± 141784.630op/s 255862865.516op/s 255991307.426op/s 256034135.792op/s 256962730.660op/s 0.48% 0.501 4.714 0.08% 15240.092op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 76.749µs 77.535µs ± 0.326µs 77.511µs ± 0.212µs 77.745µs 78.098µs 78.411µs 78.476µs 1.24% 0.351 -0.068 0.42% 0.023µs 1 200
credit_card/is_card_number/378282246310005 throughput 12742800.984op/s 12897590.356op/s ± 54161.217op/s 12901337.874op/s ± 35303.228op/s 12934005.765op/s 12972308.899op/s 13000821.261op/s 13029479.146op/s 0.99% -0.328 -0.086 0.42% 3829.776op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 51.730µs 51.792µs ± 0.055µs 51.781µs ± 0.021µs 51.810µs 51.869µs 51.890µs 52.370µs 1.14% 6.016 59.027 0.11% 0.004µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19094920.456op/s 19308112.192op/s ± 20394.645op/s 19312154.802op/s ± 7949.804op/s 19318701.110op/s 19327161.822op/s 19330612.900op/s 19331086.609op/s 0.10% -5.937 57.899 0.11% 1442.119op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.087µs 6.253µs ± 0.067µs 6.250µs ± 0.050µs 6.304µs 6.355µs 6.391µs 6.412µs 2.60% -0.095 -0.380 1.07% 0.005µs 1 200
credit_card/is_card_number/x371413321323331 throughput 155954788.951op/s 159950878.932op/s ± 1714393.269op/s 160005329.046op/s ± 1279224.799op/s 161231428.032op/s 163094795.853op/s 163728751.086op/s 164283322.503op/s 2.67% 0.147 -0.352 1.07% 121225.911op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.891µs 3.910µs ± 0.003µs 3.910µs ± 0.002µs 3.912µs 3.915µs 3.917µs 3.918µs 0.21% -0.897 6.805 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255210345.084op/s 255732598.728op/s ± 196934.726op/s 255750935.623op/s ± 109744.454op/s 255845114.828op/s 256001047.454op/s 256062308.964op/s 256978616.346op/s 0.48% 0.915 6.925 0.08% 13925.388op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 65.906µs 66.236µs ± 0.235µs 66.186µs ± 0.105µs 66.296µs 66.732µs 67.079µs 67.544µs 2.05% 2.253 6.728 0.35% 0.017µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 14805110.685op/s 15097770.974op/s ± 53084.575op/s 15108935.425op/s ± 23976.776op/s 15131242.499op/s 15147987.794op/s 15160500.092op/s 15173072.349op/s 0.42% -2.215 6.481 0.35% 3753.646op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 59.468µs 59.619µs ± 0.057µs 59.614µs ± 0.032µs 59.648µs 59.731µs 59.759µs 59.774µs 0.27% 0.355 0.318 0.09% 0.004µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 16729793.422op/s 16773169.923op/s ± 15945.853op/s 16774561.378op/s ± 8994.933op/s 16782401.974op/s 16797314.364op/s 16808553.234op/s 16815709.236op/s 0.25% -0.349 0.315 0.09% 1127.542op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.892µs 3.910µs ± 0.003µs 3.910µs ± 0.002µs 3.911µs 3.915µs 3.917µs 3.918µs 0.23% -0.371 5.112 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255205778.864op/s 255755731.738op/s ± 198193.557op/s 255781238.452op/s ± 111729.860op/s 255883008.448op/s 255992487.164op/s 256071762.053op/s 256919681.665op/s 0.45% 0.387 5.199 0.08% 14014.401op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 56.181µs 56.345µs ± 0.145µs 56.312µs ± 0.058µs 56.380µs 56.667µs 56.947µs 57.000µs 1.22% 2.348 6.497 0.26% 0.010µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17543925.357op/s 17747963.348op/s ± 45486.458op/s 17758164.437op/s ± 18158.895op/s 17775323.852op/s 17793370.813op/s 17796703.037op/s 17799565.691op/s 0.23% -2.326 6.377 0.26% 3216.378op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 51.730µs 51.799µs ± 0.033µs 51.797µs ± 0.023µs 51.820µs 51.854µs 51.882µs 51.920µs 0.24% 0.495 0.299 0.06% 0.002µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19260540.207op/s 19305482.101op/s ± 12348.012op/s 19306038.477op/s ± 8719.701op/s 19314799.522op/s 19323617.413op/s 19327821.722op/s 19331321.614op/s 0.13% -0.491 0.291 0.06% 873.136op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.062µs 6.252µs ± 0.069µs 6.250µs ± 0.050µs 6.302µs 6.368µs 6.402µs 6.438µs 3.00% 0.084 -0.162 1.10% 0.005µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 155328241.650op/s 159975798.060op/s ± 1763894.631op/s 159992700.809op/s ± 1290661.183op/s 161266591.249op/s 162874820.639op/s 163768756.270op/s 164972571.756op/s 3.11% -0.024 -0.158 1.10% 124726.185op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.910µs; 3.911µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255694942.796op/s; 255748490.231op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [77.481µs; 77.618µs] or [-0.088%; +0.088%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12884304.359op/s; 12906679.506op/s] or [-0.087%; +0.087%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [71.363µs; 71.482µs] or [-0.084%; +0.084%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13990316.328op/s; 14012955.196op/s] or [-0.081%; +0.081%] None None None
credit_card/is_card_number/37828224631 execution_time [3.910µs; 3.911µs] or [-0.012%; +0.012%] None None None
credit_card/is_card_number/37828224631 throughput [255681983.356op/s; 255741723.419op/s] or [-0.012%; +0.012%] None None None
credit_card/is_card_number/378282246310005 execution_time [77.490µs; 77.580µs] or [-0.058%; +0.058%] None None None
credit_card/is_card_number/378282246310005 throughput [12890084.132op/s; 12905096.580op/s] or [-0.058%; +0.058%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [51.784µs; 51.799µs] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19305285.690op/s; 19310938.693op/s] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.243µs; 6.262µs] or [-0.148%; +0.148%] None None None
credit_card/is_card_number/x371413321323331 throughput [159713280.513op/s; 160188477.351op/s] or [-0.149%; +0.149%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.910µs; 3.911µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ throughput [255705305.469op/s; 255759891.987op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [66.203µs; 66.268µs] or [-0.049%; +0.049%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15090413.962op/s; 15105127.985op/s] or [-0.049%; +0.049%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [59.611µs; 59.627µs] or [-0.013%; +0.013%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [16770959.981op/s; 16775379.864op/s] or [-0.013%; +0.013%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.910µs; 3.910µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255728264.017op/s; 255783199.459op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [56.325µs; 56.365µs] or [-0.036%; +0.036%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [17741659.362op/s; 17754267.334op/s] or [-0.036%; +0.036%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [51.794µs; 51.803µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19303770.785op/s; 19307193.416op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.242µs; 6.261µs] or [-0.153%; +0.153%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [159731339.229op/s; 160220256.892op/s] or [-0.153%; +0.153%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 146.679µs 147.952µs ± 0.367µs 147.888µs ± 0.207µs 148.121µs 148.517µs 149.102µs 150.118µs 1.51% 1.604 6.941 0.25% 0.026µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [147.901µs; 148.003µs] or [-0.034%; +0.034%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.543µs 33.207µs ± 1.228µs 32.639µs ± 0.054µs 32.738µs 35.855µs 35.907µs 36.447µs 11.67% 1.697 0.921 3.69% 0.087µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.037µs; 33.377µs] or [-0.512%; +0.512%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.512µs 26.241µs ± 10.566µs 18.481µs ± 0.188µs 34.646µs 44.775µs 54.858µs 93.240µs 404.53% 1.801 6.734 40.16% 0.747µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.777µs; 27.706µs] or [-5.580%; +5.580%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 02d3083e 1746171480 gleocadie/fix-normalization-regression
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 462.212µs 467.676µs ± 1.955µs 467.862µs ± 1.168µs 468.915µs 469.755µs 470.782µs 478.708µs 2.32% 0.852 6.408 0.42% 0.138µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 2088957.026op/s 2138271.524op/s ± 8907.445op/s 2137383.834op/s ± 5336.239op/s 2143217.220op/s 2153114.053op/s 2159979.871op/s 2163510.124op/s 1.22% -0.758 5.952 0.42% 629.851op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 452.810µs 453.456µs ± 0.373µs 453.448µs ± 0.264µs 453.681µs 454.122µs 454.611µs 454.646µs 0.26% 0.675 0.443 0.08% 0.026µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2199512.500op/s 2205286.353op/s ± 1814.968op/s 2205324.171op/s ± 1283.118op/s 2206644.538op/s 2207871.170op/s 2208391.637op/s 2208433.987op/s 0.14% -0.670 0.432 0.08% 128.338op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 179.706µs 180.209µs ± 0.281µs 180.167µs ± 0.200µs 180.386µs 180.710µs 180.933µs 181.172µs 0.56% 0.655 0.155 0.16% 0.020µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5519621.176op/s 5549127.140op/s ± 8636.292op/s 5550419.913op/s ± 6162.855op/s 5555874.002op/s 5560564.208op/s 5563138.357op/s 5564632.391op/s 0.26% -0.647 0.137 0.16% 610.678op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 41.155µs 41.299µs ± 0.148µs 41.288µs ± 0.039µs 41.326µs 41.371µs 41.437µs 42.720µs 3.47% 8.013 72.159 0.36% 0.010µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 23408132.706op/s 24213944.640op/s ± 84247.990op/s 24220147.096op/s ± 22983.097op/s 24244401.358op/s 24271653.110op/s 24286411.277op/s 24298321.274op/s 0.32% -7.906 70.851 0.35% 5957.233op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.212µs 48.321µs ± 0.049µs 48.316µs ± 0.027µs 48.342µs 48.414µs 48.462µs 48.621µs 0.63% 1.574 6.379 0.10% 0.003µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20567351.575op/s 20695123.581op/s ± 21099.463op/s 20697210.516op/s ± 11573.460op/s 20708770.771op/s 20721834.148op/s 20732209.851op/s 20741733.942op/s 0.22% -1.556 6.261 0.10% 1491.957op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [467.405µs; 467.946µs] or [-0.058%; +0.058%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [2137037.038op/s; 2139506.011op/s] or [-0.058%; +0.058%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [453.404µs; 453.508µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2205034.816op/s; 2205537.890op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [180.170µs; 180.248µs] or [-0.022%; +0.022%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5547930.233op/s; 5550324.047op/s] or [-0.022%; +0.022%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [41.279µs; 41.320µs] or [-0.050%; +0.050%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [24202268.679op/s; 24225620.601op/s] or [-0.048%; +0.048%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [48.314µs; 48.327µs] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20692199.399op/s; 20698047.764op/s] or [-0.014%; +0.014%] None None None

Baseline

Omitted due to size.

pr-commenter[bot] avatar May 02 '25 07:05 pr-commenter[bot]

Codecov Report

Attention: Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.

Project coverage is 71.28%. Comparing base (540ea1e) to head (12ba522).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1048   +/-   ##
=======================================
  Coverage   71.28%   71.28%           
=======================================
  Files         329      329           
  Lines       49225    49229    +4     
=======================================
+ Hits        35088    35093    +5     
+ Misses      14137    14136    -1     
Components Coverage Δ
datadog-crashtracker 42.80% <0.00%> (-0.05%) :arrow_down:
datadog-crashtracker-ffi 6.30% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 90.36% <ø> (ø)
data-pipeline-ffi 89.02% <ø> (ø)
ddcommon 78.29% <ø> (ø)
ddcommon-ffi 66.37% <ø> (ø)
ddtelemetry 60.19% <ø> (ø)
ddtelemetry-ffi 21.32% <ø> (ø)
dogstatsd-client 82.57% <ø> (ø)
datadog-ipc 82.54% <ø> (-0.19%) :arrow_down:
datadog-profiling 77.49% <ø> (ø)
datadog-profiling-ffi 62.12% <ø> (ø)
datadog-sidecar 41.90% <ø> (ø)
datdog-sidecar-ffi 7.73% <ø> (ø)
spawn-worker 54.37% <ø> (ø)
tinybytes 88.94% <ø> (ø)
datadog-trace-normalization 98.24% <ø> (ø)
datadog-trace-obfuscation 94.16% <ø> (ø)
datadog-trace-protobuf 78.50% <ø> (ø)
datadog-trace-utils 89.26% <ø> (ø)
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar May 02 '25 07:05 codecov-commenter

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 66.03 MB 66.03 MB -0% (-128 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 20.31 MB 20.31 MB -0% (-88 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.03 MB 8.03 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 22.70 MB 22.70 MB -0% (-64 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.00 MB 8.00 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 72.62 MB 72.62 MB -0% (-104 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 16.71 MB 16.71 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 61.92 KB 61.92 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 112.78 MB 112.78 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 632.17 MB 632.17 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 4.99 MB 4.99 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 61.92 KB 61.92 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 15.87 MB 15.87 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 26.42 MB 26.42 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 14.12 MB 14.12 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 62.87 KB 62.87 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 114.49 MB 114.49 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 625.10 MB 625.10 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 3.78 MB 3.78 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 62.87 KB 62.87 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 16.50 MB 16.50 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 24.34 MB 24.34 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 57.50 MB 57.50 MB -0% (-96 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.42 MB 8.42 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 18.71 MB 18.71 MB -0% (-48 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 68.59 MB 68.59 MB -0% (-88 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 8.42 MB 8.42 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 20.89 MB 20.89 MB -0% (-40 B) 👌

r1viollet avatar May 02 '25 08:05 r1viollet