libdatadog icon indicating copy to clipboard operation
libdatadog copied to clipboard
verified publisher icon DataDog

Remote configuration support

Open bwoebi opened this issue 1 year ago • 6 comments

Also including Dynamic Configuration, but it's a small part, just 100 lines of code, describing the structures it contains.

As to more specific changes to current code:

  • The sidecar is now per user (on windows it already uses the ConsoleSession), instead of per system.
    • On Windows it was necessary as memory can only easily be shared within a ConsoleSession.
    • On Linux it's now necessary as the notification mechanism for changes to the remote config are distributed via signals, which require either same euid or root-like capabilities. Note AsBytes in ddcommon currently requires &'a AsBytes<'a>, while only &AsBytes<'a> (independent lifetime) is required.

General notes:

  • Code in remote-config/ is meant to be reusable and also accessible via FFI as needed. For most users the bare ConfigFetcher should be enough, and for some the SharedFetcher is needed. Fewest should actually need the MultiTargetFetcher. A FFi interface hasn't been defined yet as it wasn't needed:
    • ConfigFetcher in its simplest usage is just one function with two callbacks (store, update) essentially. StoredFile should be a wrapper for a type native to the FFI user (a pointer probably).
    • SharedFetcher is similar, but presents a run-loop. To be used in anyway multi-threaded applications, now three callbacks: store, update and on_fetched.
  • The sidecar will notify participating processes whenever remote config changes.
  • The sidecar uses shared memory to transmit the stuff received by the remote-config stuff.

bwoebi avatar Jun 14 '24 14:06 bwoebi

Codecov Report

Attention: Patch coverage is 73.92501% with 758 lines in your changes missing coverage. Please review.

Project coverage is 71.83%. Comparing base (dd36a81) to head (22cbb0b). Report is 101 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #488      +/-   ##
==========================================
+ Coverage   71.17%   71.83%   +0.66%     
==========================================
  Files         220      237      +17     
  Lines       30000    32871    +2871     
==========================================
+ Hits        21351    23612    +2261     
- Misses       8649     9259     +610
Components Coverage Δ
crashtracker 21.25% <ø> (+0.05%) :arrow_up:
datadog-alloc 98.73% <ø> (ø)
data-pipeline 50.00% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.11% <0.00%> (-0.67%) :arrow_down:
ddcommon-ffi 68.11% <0.00%> (-1.61%) :arrow_down:
ddtelemetry 59.02% <ø> (ø)
ipc 84.29% <ø> (+0.10%) :arrow_up:
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 40.47% <54.82%> (+6.43%) :arrow_up:
sidecar-ffi 0.00% <0.00%> (ø)
spawn-worker 54.87% <ø> (ø)
trace-mini-agent 70.88% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <79.24%> (+0.51%) :arrow_up:
trace-utils 92.97% <ø> (-0.43%) :arrow_down:

codecov-commenter avatar Jun 14 '24 18:06 codecov-commenter

Can you add some examples to the codebase/comments so example/expected usage is shown in cargo doc for the remote-config library?

brettlangdon avatar Jun 24 '24 12:06 brettlangdon

@brettlangdon Implemented some helper functionality to auto-parse, diff and locally store remote config files. These all weren't needed for the purposes of the sidecar (sidecar doesn't parse most things, but just writes them to shared memory, no explicit backing storage needed; diffing happened based on shared memory contents in target processes.), but they are probably useful for most implementers.

Please have a look at examples/remote_config_fetch.rs.

bwoebi avatar Jun 25 '24 16:06 bwoebi

Please split this up in separate PR, nobody can review this much code at once.

Also RC should be in the code owners of the RC implementation like for other tracers

mellon85 avatar Jul 03 '24 14:07 mellon85

Please consider the files in remote-config individually: fetch/fetcher.rs as the primary RC client. fetch/shared.rs for sharing file storage. fetch/multitarget.rs for multiple targets. fetch/single.rs for users just needing a single client, packaged in an a bit nicer API. Uses file_change_tracker.rs and file_storage.rs

Then parse.rs to process remote config paths / provide an entry point to the individual products.

I don't intend to split this RC beyond the point where I can do minimal end-to-end testing.

EDIT: Updated CODEOWNERS accordingly.

bwoebi avatar Jul 03 '24 15:07 bwoebi

Benchmarks

Comparison

Benchmark execution time: 2024-08-16 14:08:12

Comparing candidate commit 22cbb0b in PR branch bob/remote-config with baseline commit dd36a81 in branch main.

Found 30 performance improvements and 8 performance regressions! Performance is the same for 12 metrics, 2 unstable metrics.

scenario:benching string interning on wordpress profile

  • 🟥 execution_time [+3.350µs; +3.437µs] or [+2.483%; +2.548%]

scenario:credit_card/is_card_number/

  • 🟩 execution_time [-802.546ns; -802.284ns] or [-33.218%; -33.207%]
  • 🟩 throughput [+205792636.645op/s; +205855155.356op/s] or [+49.720%; +49.735%]

scenario:credit_card/is_card_number/ 3782-8224-6310-005

  • 🟩 execution_time [-3.203µs; -2.565µs] or [-2.973%; -2.380%]
  • 🟩 throughput [+224153.385op/s; +278462.372op/s] or [+2.415%; +3.000%]

scenario:credit_card/is_card_number/ 378282246310005

  • 🟩 execution_time [-7.602µs; -6.942µs] or [-7.271%; -6.641%]
  • 🟩 throughput [+680622.834op/s; +740678.106op/s] or [+7.112%; +7.740%]

scenario:credit_card/is_card_number/37828224631

  • 🟩 execution_time [-803.803ns; -802.127ns] or [-33.259%; -33.189%]
  • 🟩 throughput [+205662367.850op/s; +206074442.350op/s] or [+49.705%; +49.804%]

scenario:credit_card/is_card_number/378282246310005

  • 🟩 execution_time [-6.871µs; -5.993µs] or [-6.738%; -5.877%]
  • 🟩 throughput [+609270.474op/s; +693540.668op/s] or [+6.207%; +7.066%]

scenario:credit_card/is_card_number/37828224631000521389798

  • 🟩 execution_time [-14.013µs; -13.710µs] or [-12.908%; -12.629%]
  • 🟩 throughput [+1334274.984op/s; +1361032.373op/s] or [+14.484%; +14.774%]

scenario:credit_card/is_card_number/x371413321323331

  • 🟩 execution_time [-1.106µs; -0.942µs] or [-4.647%; -3.959%]
  • 🟩 throughput [+1724911.099op/s; +2018369.948op/s] or [+4.103%; +4.801%]

scenario:credit_card/is_card_number_no_luhn/

  • 🟩 execution_time [-802.619ns; -802.379ns] or [-33.220%; -33.210%]
  • 🟩 throughput [+205821805.251op/s; +205882226.241op/s] or [+49.728%; +49.742%]

scenario:credit_card/is_card_number_no_luhn/ 3782-8224-6310-005

  • 🟩 execution_time [-3.407µs; -3.258µs] or [-3.789%; -3.623%]
  • 🟩 throughput [+418425.140op/s; +437438.107op/s] or [+3.763%; +3.934%]

scenario:credit_card/is_card_number_no_luhn/ 378282246310005

  • 🟩 execution_time [-5.586µs; -5.153µs] or [-6.536%; -6.029%]
  • 🟩 throughput [+751566.400op/s; +811362.375op/s] or [+6.422%; +6.933%]

scenario:credit_card/is_card_number_no_luhn/37828224631

  • 🟩 execution_time [-802.590ns; -802.355ns] or [-33.219%; -33.210%]
  • 🟩 throughput [+205818236.233op/s; +205877221.815op/s] or [+49.726%; +49.740%]

scenario:credit_card/is_card_number_no_luhn/378282246310005

  • 🟩 execution_time [-6.559µs; -6.270µs] or [-7.751%; -7.409%]
  • 🟩 throughput [+946970.228op/s; +989221.607op/s] or [+8.013%; +8.371%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

  • 🟩 execution_time [-13.745µs; -13.409µs] or [-12.711%; -12.401%]
  • 🟩 throughput [+1312182.384op/s; +1341857.663op/s] or [+14.187%; +14.508%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟩 execution_time [-993.146ns; -820.553ns] or [-4.188%; -3.460%]
  • 🟩 throughput [+1500695.452op/s; +1810782.911op/s] or [+3.557%; +4.292%]

scenario:normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...

  • 🟥 execution_time [+32.046µs; +32.603µs] or [+11.354%; +11.551%]
  • 🟥 throughput [-367568.277op/s; -360841.404op/s] or [-10.374%; -10.184%]

scenario:normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...

  • 🟥 execution_time [+111.689µs; +111.955µs] or [+21.992%; +22.044%]
  • 🟥 throughput [-355679.274op/s; -354926.070op/s] or [-18.064%; -18.026%]

scenario:normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters

  • 🟥 execution_time [+10.240µs; +10.328µs] or [+5.646%; +5.695%]
  • 🟥 throughput [-297116.470op/s; -294617.273op/s] or [-5.389%; -5.343%]

scenario:normalization/normalize_service/normalize_service/test_ASCII

  • 🟩 execution_time [-1.893µs; -1.802µs] or [-3.676%; -3.501%]
  • 🟩 throughput [+705274.388op/s; +741032.741op/s] or [+3.631%; +3.815%]

scenario:redis/obfuscate_redis_string

  • 🟥 execution_time [+1.521µs; +1.843µs] or [+3.989%; +4.833%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 618.506µs 619.691µs ± 0.884µs 619.414µs ± 0.448µs 620.250µs 621.559µs 622.188µs 623.349µs 0.64% 1.283 1.310 0.14% 0.063µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1604237.068op/s 1613710.926op/s ± 2298.191op/s 1614428.687op/s ± 1169.286op/s 1615391.711op/s 1616100.890op/s 1616466.569op/s 1616798.842op/s 0.15% -1.276 1.281 0.14% 162.507op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 387.151µs 389.682µs ± 1.224µs 389.698µs ± 0.819µs 390.416µs 392.100µs 392.543µs 393.005µs 0.85% 0.362 -0.252 0.31% 0.087µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2544493.652op/s 2566219.442op/s ± 8049.701op/s 2566089.642op/s ± 5396.354op/s 2571805.993op/s 2577993.488op/s 2581306.580op/s 2582974.231op/s 0.66% -0.346 -0.269 0.31% 569.200op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 190.941µs 191.647µs ± 0.260µs 191.640µs ± 0.177µs 191.804µs 192.155µs 192.286µs 192.450µs 0.42% 0.330 0.226 0.14% 0.018µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5196160.384op/s 5217946.400op/s ± 7069.043op/s 5218125.333op/s ± 4819.762op/s 5223186.771op/s 5228146.143op/s 5233493.880op/s 5237215.654op/s 0.37% -0.321 0.220 0.14% 499.857op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 44.955µs 45.205µs ± 0.115µs 45.209µs ± 0.086µs 45.287µs 45.389µs 45.449µs 45.494µs 0.63% 0.115 -0.714 0.25% 0.008µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 21981055.157op/s 22121696.969op/s ± 56402.312op/s 22119358.429op/s ± 42001.600op/s 22162556.600op/s 22209648.038op/s 22223660.418op/s 22244417.207op/s 0.57% -0.105 -0.720 0.25% 3988.246op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.034µs 49.637µs ± 0.255µs 49.677µs ± 0.156µs 49.806µs 50.006µs 50.183µs 50.229µs 1.11% -0.314 -0.221 0.51% 0.018µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19908992.753op/s 20146847.180op/s ± 103686.106op/s 20130233.261op/s ± 63099.364op/s 20218824.107op/s 20334190.085op/s 20384576.117op/s 20393818.871op/s 1.31% 0.339 -0.214 0.51% 7331.715op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [619.568µs; 619.813µs] or [-0.020%; +0.020%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1613392.419op/s; 1614029.433op/s] or [-0.020%; +0.020%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [389.513µs; 389.852µs] or [-0.044%; +0.044%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2565103.831op/s; 2567335.053op/s] or [-0.043%; +0.043%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [191.611µs; 191.683µs] or [-0.019%; +0.019%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5216966.698op/s; 5218926.101op/s] or [-0.019%; +0.019%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [45.189µs; 45.221µs] or [-0.035%; +0.035%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22113880.151op/s; 22129513.787op/s] or [-0.035%; +0.035%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.602µs; 49.672µs] or [-0.071%; +0.071%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20132477.283op/s; 20161217.077op/s] or [-0.071%; +0.071%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 312.028µs 314.571µs ± 0.755µs 314.639µs ± 0.493µs 315.119µs 315.715µs 316.013µs 316.110µs 0.47% -0.459 0.091 0.24% 0.053µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3163451.910op/s 3178947.648op/s ± 7640.125op/s 3178241.077op/s ± 4975.401op/s 3183614.280op/s 3192265.093op/s 3198133.790op/s 3204840.591op/s 0.84% 0.472 0.112 0.24% 540.238op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 27.929µs 27.988µs ± 0.050µs 27.967µs ± 0.018µs 28.019µs 28.082µs 28.133µs 28.144µs 0.63% 1.208 0.378 0.18% 0.004µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 35531806.946op/s 35730200.148op/s ± 63115.817op/s 35756562.447op/s ± 23194.571op/s 35776126.075op/s 35788689.195op/s 35796461.098op/s 35804682.702op/s 0.13% -1.203 0.362 0.18% 4462.962op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 16.699µs 16.728µs ± 0.021µs 16.718µs ± 0.016µs 16.747µs 16.758µs 16.767µs 16.785µs 0.40% 0.347 -1.220 0.12% 0.001µs 1 200
normalization/normalize_name/normalize_name/good throughput 59578417.062op/s 59780389.669op/s ± 73506.789op/s 59816010.009op/s ± 56831.240op/s 59846923.741op/s 59869493.637op/s 59876846.115op/s 59882427.451op/s 0.11% -0.344 -1.226 0.12% 5197.715op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [314.467µs; 314.676µs] or [-0.033%; +0.033%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3177888.800op/s; 3180006.496op/s] or [-0.033%; +0.033%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [27.981µs; 27.994µs] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [35721452.903op/s; 35738947.393op/s] or [-0.024%; +0.024%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.725µs; 16.731µs] or [-0.017%; +0.017%] None None None
normalization/normalize_name/normalize_name/good throughput [59770202.335op/s; 59790577.003op/s] or [-0.017%; +0.017%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 39.409µs 39.814µs ± 0.657µs 39.516µs ± 0.071µs 39.663µs 41.121µs 41.212µs 43.396µs 9.82% 2.043 4.122 1.65% 0.046µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [39.723µs; 39.905µs] or [-0.229%; +0.229%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 137.916µs 138.270µs ± 0.239µs 138.233µs ± 0.096µs 138.338µs 138.598µs 139.174µs 140.002µs 1.28% 3.453 19.095 0.17% 0.017µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [138.237µs; 138.303µs] or [-0.024%; +0.024%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.656µs 2.710µs ± 0.026µs 2.716µs ± 0.011µs 2.720µs 2.759µs 2.764µs 2.766µs 1.87% -0.176 -0.064 0.94% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.706µs; 2.713µs] or [-0.130%; +0.130%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 299.727ns 310.493ns ± 13.289ns 303.929ns ± 2.760ns 313.284ns 335.892ns 349.785ns 351.260ns 15.57% 1.522 1.128 4.27% 0.940ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [308.651ns; 312.334ns] or [-0.593%; +0.593%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.613µs 1.614µs ± 0.000µs 1.614µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.615µs 0.08% 0.672 -0.073 0.03% 0.000µs 1 200
credit_card/is_card_number/ throughput 619227101.440op/s 619725746.328op/s ± 175630.710op/s 619745454.294op/s ± 131200.462op/s 619874061.820op/s 619954006.137op/s 619967657.561op/s 620047550.322op/s 0.05% -0.671 -0.076 0.03% 12418.967op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 104.101µs 104.880µs ± 0.425µs 104.861µs ± 0.186µs 105.039µs 105.476µs 106.065µs 108.496µs 3.47% 3.504 25.705 0.40% 0.030µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 9216911.958op/s 9534880.739op/s ± 38087.670op/s 9536391.797op/s ± 16886.355op/s 9554742.529op/s 9581026.602op/s 9602342.137op/s 9606010.481op/s 0.73% -3.322 23.747 0.40% 2693.205op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 96.237µs 97.272µs ± 0.469µs 97.200µs ± 0.274µs 97.499µs 98.128µs 98.753µs 99.058µs 1.91% 1.019 1.622 0.48% 0.033µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 10095077.184op/s 10280671.665op/s ± 49334.839op/s 10288064.827op/s ± 29059.777op/s 10313092.602op/s 10346533.101op/s 10365867.582op/s 10390966.094op/s 1.00% -0.982 1.513 0.48% 3488.500op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.613µs 1.614µs ± 0.003µs 1.614µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.659µs 2.83% 13.517 185.291 0.20% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 602664383.323op/s 619637200.268op/s ± 1222319.434op/s 619709894.232op/s ± 147132.615op/s 619882010.390op/s 619981710.444op/s 620012484.501op/s 620096033.847op/s 0.06% -13.489 184.765 0.20% 86431.036op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 94.396µs 95.544µs ± 0.528µs 95.504µs ± 0.373µs 95.889µs 96.461µs 96.776µs 97.056µs 1.62% 0.324 -0.293 0.55% 0.037µs 1 200
credit_card/is_card_number/378282246310005 throughput 10303325.029op/s 10466675.676op/s ± 57768.239op/s 10470751.763op/s ± 40935.281op/s 10509025.990op/s 10555552.270op/s 10579279.974op/s 10593717.925op/s 1.17% -0.298 -0.316 0.55% 4084.831op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 94.096µs 94.699µs ± 0.381µs 94.685µs ± 0.256µs 94.924µs 95.280µs 95.620µs 96.953µs 2.40% 1.309 5.161 0.40% 0.027µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 10314223.801op/s 10559903.805op/s ± 42269.565op/s 10561294.461op/s ± 28550.005op/s 10591075.443op/s 10617952.994op/s 10626672.299op/s 10627461.360op/s 0.63% -1.246 4.743 0.40% 2988.910op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 22.445µs 22.775µs ± 0.226µs 22.767µs ± 0.155µs 22.911µs 23.197µs 23.407µs 23.467µs 3.07% 0.706 0.143 0.99% 0.016µs 1 200
credit_card/is_card_number/x371413321323331 throughput 42613728.536op/s 43911169.475op/s ± 432134.743op/s 43923827.325op/s ± 300940.638op/s 44230232.738op/s 44521811.214op/s 44549337.120op/s 44553122.561op/s 1.43% -0.658 0.045 0.98% 30556.541op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.613µs 1.614µs ± 0.000µs 1.614µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.615µs 0.09% 0.541 -0.118 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 619210635.317op/s 619750417.613op/s ± 178653.814op/s 619764921.309op/s ± 134324.561op/s 619897261.881op/s 619984643.827op/s 620026588.761op/s 620131150.034op/s 0.06% -0.540 -0.121 0.03% 12632.732op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 85.883µs 86.599µs ± 0.360µs 86.550µs ± 0.175µs 86.728µs 87.231µs 87.796µs 88.578µs 2.34% 1.613 5.258 0.41% 0.025µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 11289476.184op/s 11547690.438op/s ± 47656.789op/s 11554000.636op/s ± 23391.437op/s 11577238.281op/s 11608546.540op/s 11626839.398op/s 11643694.127op/s 0.78% -1.556 4.947 0.41% 3369.844op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 79.106µs 80.101µs ± 0.544µs 80.025µs ± 0.362µs 80.457µs 80.909µs 81.640µs 82.458µs 3.04% 0.812 1.468 0.68% 0.038µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 12127344.883op/s 12484853.741op/s ± 84273.606op/s 12496090.605op/s ± 56552.637op/s 12548113.940op/s 12597158.396op/s 12635212.481op/s 12641294.274op/s 1.16% -0.756 1.275 0.67% 5959.044op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.613µs 1.614µs ± 0.000µs 1.614µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.615µs 0.09% 0.510 -0.107 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 619193066.636op/s 619750948.669op/s ± 173288.458op/s 619768174.549op/s ± 121022.944op/s 619887205.936op/s 619985515.730op/s 620026447.100op/s 620119076.472op/s 0.06% -0.509 -0.110 0.03% 12253.344op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 77.075µs 78.218µs ± 0.483µs 78.204µs ± 0.341µs 78.529µs 78.994µs 79.530µs 79.606µs 1.79% 0.333 0.013 0.62% 0.034µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 12561800.624op/s 12785335.155op/s ± 78718.882op/s 12787041.670op/s ± 55709.778op/s 12845481.355op/s 12906432.186op/s 12926919.637op/s 12974312.483op/s 1.46% -0.298 -0.032 0.61% 5566.266op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 94.099µs 94.555µs ± 0.381µs 94.463µs ± 0.255µs 94.769µs 95.259µs 95.558µs 95.834µs 1.45% 0.914 0.199 0.40% 0.027µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 10434695.639op/s 10575992.915op/s ± 42461.254op/s 10586171.301op/s ± 28598.622op/s 10612447.074op/s 10623904.316op/s 10625564.045op/s 10627082.056op/s 0.39% -0.897 0.152 0.40% 3002.464op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 22.444µs 22.808µs ± 0.226µs 22.790µs ± 0.153µs 22.941µs 23.206µs 23.484µs 23.498µs 3.10% 0.780 0.414 0.99% 0.016µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 42557710.636op/s 43848413.328op/s ± 431791.879op/s 43878771.203op/s ± 294288.253op/s 44174218.693op/s 44435928.573op/s 44540549.830op/s 44554494.725op/s 1.54% -0.727 0.293 0.98% 30532.297op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.614µs; 1.614µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ throughput [619701405.601op/s; 619750087.056op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [104.821µs; 104.939µs] or [-0.056%; +0.056%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [9529602.154op/s; 9540159.323op/s] or [-0.055%; +0.055%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [97.207µs; 97.337µs] or [-0.067%; +0.067%] None None None
credit_card/is_card_number/ 378282246310005 throughput [10273834.330op/s; 10287508.999op/s] or [-0.067%; +0.067%] None None None
credit_card/is_card_number/37828224631 execution_time [1.613µs; 1.614µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/37828224631 throughput [619467798.550op/s; 619806601.986op/s] or [-0.027%; +0.027%] None None None
credit_card/is_card_number/378282246310005 execution_time [95.471µs; 95.617µs] or [-0.077%; +0.077%] None None None
credit_card/is_card_number/378282246310005 throughput [10458669.554op/s; 10474681.799op/s] or [-0.076%; +0.076%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [94.647µs; 94.752µs] or [-0.056%; +0.056%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [10554045.649op/s; 10565761.960op/s] or [-0.055%; +0.055%] None None None
credit_card/is_card_number/x371413321323331 execution_time [22.744µs; 22.807µs] or [-0.137%; +0.137%] None None None
credit_card/is_card_number/x371413321323331 throughput [43851279.756op/s; 43971059.195op/s] or [-0.136%; +0.136%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.613µs; 1.614µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ throughput [619725657.912op/s; 619775177.313op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [86.549µs; 86.649µs] or [-0.058%; +0.058%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [11541085.665op/s; 11554295.210op/s] or [-0.057%; +0.057%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [80.025µs; 80.176µs] or [-0.094%; +0.094%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [12473174.229op/s; 12496533.252op/s] or [-0.094%; +0.094%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.613µs; 1.614µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [619726932.555op/s; 619774964.783op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [78.151µs; 78.284µs] or [-0.085%; +0.085%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12774425.475op/s; 12796244.835op/s] or [-0.085%; +0.085%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [94.502µs; 94.608µs] or [-0.056%; +0.056%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [10570108.193op/s; 10581877.636op/s] or [-0.056%; +0.056%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [22.777µs; 22.839µs] or [-0.137%; +0.137%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [43788571.127op/s; 43908255.530op/s] or [-0.136%; +0.136%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 73.318µs 73.528µs ± 0.176µs 73.482µs ± 0.051µs 73.543µs 73.829µs 73.905µs 75.230µs 2.38% 4.987 42.113 0.24% 0.012µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [73.504µs; 73.553µs] or [-0.033%; +0.033%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.114µs 22.362µs ± 13.582µs 17.385µs ± 0.103µs 17.519µs 54.986µs 58.069µs 125.691µs 622.99% 3.514 17.047 60.58% 0.960µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [20.480µs; 24.245µs] or [-8.417%; +8.417%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.147µs 1.343µs ± 0.061µs 1.340µs ± 0.047µs 1.388µs 1.436µs 1.453µs 1.454µs 8.44% -0.508 0.665 4.54% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.335µs; 1.352µs] or [-0.630%; +0.630%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 22cbb0b 1723816693 bob/remote-config
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.432µs 2.900µs ± 1.395µs 2.737µs ± 0.020µs 2.758µs 2.991µs 12.697µs 15.706µs 473.76% 7.946 63.087 47.99% 0.099µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.706µs; 3.093µs] or [-6.667%; +6.667%] None None None

Baseline

Omitted due to size.

pr-commenter[bot] avatar Jul 08 '24 15:07 pr-commenter[bot]