libdatadog icon indicating copy to clipboard operation
libdatadog copied to clipboard

feat(data-pipeline-ffi): add functions to manipulate span from C

Open Leiyks opened this issue 8 months ago • 5 comments

What does this PR do?

This PR add functions to create and manipulate field of Rust Span struct from C code.

Motivation

The replacement of PHP Span to Rust Span in the PHP Tracer.

Additional Notes

As I am using macros to create the functions, the tests need to be run on a nightly version of Rust.

How to test the change?

Tests have been added, and this is linked to another one in the PHP tracer. (TODO)

Leiyks avatar Apr 02 '25 15:04 Leiyks

Benchmarks

Comparison

Benchmark execution time: 2025-07-23 18:27:24

Comparing candidate commit 08805b5 in PR branch leiyks-add-span-conversion-functions with baseline commit 36d16a2 in branch main.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

scenario:benching deserializing traces from msgpack to their internal representation

  • 🟩 execution_time [-13.899ms; -13.265ms] or [-18.819%; -17.962%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.250µs 3.240µs ± 1.477µs 3.015µs ± 0.025µs 3.044µs 3.677µs 14.057µs 15.561µs 416.18% 7.441 56.318 45.49% 0.104µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.035µs; 3.445µs] or [-6.320%; +6.320%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.996µs 5.084µs ± 0.052µs 5.082µs ± 0.048µs 5.136µs 5.168µs 5.171µs 5.176µs 1.84% 0.322 -1.331 1.02% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.077µs; 5.091µs] or [-0.141%; +0.141%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 205.911µs 206.364µs ± 0.271µs 206.305µs ± 0.157µs 206.507µs 206.777µs 207.143µs 207.903µs 0.77% 1.755 6.312 0.13% 0.019µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4809932.276op/s 4845823.786op/s ± 6360.410op/s 4847198.931op/s ± 3690.686op/s 4850171.486op/s 4853244.639op/s 4855150.305op/s 4856467.114op/s 0.19% -1.734 6.179 0.13% 449.749op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.199µs 18.275µs ± 0.076µs 18.253µs ± 0.019µs 18.284µs 18.460µs 18.627µs 18.774µs 2.85% 3.508 15.252 0.42% 0.005µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 53266139.753op/s 54720873.600op/s ± 224662.492op/s 54786824.709op/s ± 56037.994op/s 54824363.994op/s 54912328.979op/s 54934776.039op/s 54947101.773op/s 0.29% -3.447 14.717 0.41% 15886.037op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.939µs 11.000µs ± 0.030µs 10.999µs ± 0.021µs 11.017µs 11.053µs 11.071µs 11.084µs 0.77% 0.311 -0.172 0.27% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 90218595.556op/s 90912746.343op/s ± 249754.978op/s 90913692.973op/s ± 171377.864op/s 91110339.994op/s 91265802.817op/s 91401556.459op/s 91417237.274op/s 0.55% -0.297 -0.184 0.27% 17660.344op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [206.326µs; 206.401µs] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4844942.295op/s; 4846705.278op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.264µs; 18.285µs] or [-0.058%; +0.058%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54689737.539op/s; 54752009.660op/s] or [-0.057%; +0.057%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.995µs; 11.004µs] or [-0.038%; +0.038%] None None None
normalization/normalize_name/normalize_name/good throughput [90878132.705op/s; 90947359.981op/s] or [-0.038%; +0.038%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 85.134µs 85.325µs ± 0.145µs 85.299µs ± 0.043µs 85.361µs 85.496µs 85.652µs 86.928µs 1.91% 7.110 73.664 0.17% 0.010µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [85.305µs; 85.345µs] or [-0.024%; +0.024%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.254ms 8.273ms ± 0.009ms 8.272ms ± 0.006ms 8.278ms 8.288ms 8.300ms 8.316ms 0.53% 0.925 1.904 0.11% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.272ms; 8.274ms] or [-0.015%; +0.015%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 535.741µs 536.638µs ± 0.433µs 536.616µs ± 0.211µs 536.850µs 537.162µs 537.338µs 540.727µs 0.77% 4.097 38.113 0.08% 0.031µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1849360.453op/s 1863453.773op/s ± 1498.492op/s 1863529.360op/s ± 731.606op/s 1864184.458op/s 1865420.327op/s 1866261.931op/s 1866571.947op/s 0.16% -4.041 37.451 0.08% 105.959op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 379.801µs 380.327µs ± 0.247µs 380.314µs ± 0.163µs 380.471µs 380.736µs 380.913µs 381.172µs 0.23% 0.346 -0.044 0.06% 0.017µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2623489.953op/s 2629319.199op/s ± 1709.146op/s 2629403.672op/s ± 1124.208op/s 2630533.581op/s 2632052.145op/s 2632513.433op/s 2632960.800op/s 0.14% -0.342 -0.049 0.06% 120.855op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 194.465µs 195.207µs ± 0.370µs 195.277µs ± 0.329µs 195.516µs 195.749µs 195.885µs 196.057µs 0.40% 0.022 -1.190 0.19% 0.026µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5100546.537op/s 5122772.833op/s ± 9715.115op/s 5120920.774op/s ± 8621.573op/s 5132177.891op/s 5136876.502op/s 5139124.376op/s 5142306.568op/s 0.42% -0.018 -1.192 0.19% 686.962op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 36.752µs 36.928µs ± 0.080µs 36.914µs ± 0.056µs 36.977µs 37.078µs 37.126µs 37.152µs 0.65% 0.535 -0.362 0.22% 0.006µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26916257.275op/s 27080098.861op/s ± 58955.298op/s 27089936.170op/s ± 41146.716op/s 27125326.885op/s 27158510.784op/s 27171491.638op/s 27209046.619op/s 0.44% -0.526 -0.374 0.22% 4168.769op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 45.878µs 46.018µs ± 0.065µs 46.017µs ± 0.043µs 46.053µs 46.127µs 46.195µs 46.296µs 0.61% 0.673 1.158 0.14% 0.005µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 21600288.580op/s 21730883.914op/s ± 30558.841op/s 21731083.636op/s ± 20539.899op/s 21753581.594op/s 21772697.211op/s 21788737.654op/s 21797135.033op/s 0.30% -0.662 1.122 0.14% 2160.836op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [536.578µs; 536.698µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1863246.096op/s; 1863661.449op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [380.292µs; 380.361µs] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2629082.328op/s; 2629556.070op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [195.156µs; 195.259µs] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5121426.411op/s; 5124119.254op/s] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [36.917µs; 36.939µs] or [-0.030%; +0.030%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [27071928.224op/s; 27088269.498op/s] or [-0.030%; +0.030%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [46.009µs; 46.027µs] or [-0.020%; +0.020%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21726648.753op/s; 21735119.076op/s] or [-0.019%; +0.019%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.391µs 2.420µs ± 0.016µs 2.417µs ± 0.008µs 2.427µs 2.459µs 2.466µs 2.472µs 2.25% 1.302 1.857 0.64% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.418µs; 2.422µs] or [-0.089%; +0.089%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.895µs 3.912µs ± 0.003µs 3.912µs ± 0.002µs 3.914µs 3.916µs 3.919µs 3.922µs 0.24% -0.916 8.137 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 255002288.239op/s 255609999.956op/s ± 174844.946op/s 255605068.293op/s ± 101705.728op/s 255720316.981op/s 255822124.367op/s 255902308.992op/s 256744745.719op/s 0.45% 0.936 8.253 0.07% 12363.405op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 78.900µs 80.639µs ± 0.757µs 80.621µs ± 0.510µs 81.127µs 81.912µs 82.466µs 83.017µs 2.97% 0.265 -0.080 0.94% 0.054µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12045778.232op/s 12401992.305op/s ± 116165.802op/s 12403699.486op/s ± 77908.132op/s 12488397.579op/s 12592526.391op/s 12630829.008op/s 12674202.570op/s 2.18% -0.213 -0.129 0.93% 8214.163op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 74.471µs 74.943µs ± 0.188µs 74.906µs ± 0.096µs 75.029µs 75.311µs 75.499µs 75.548µs 0.86% 0.762 0.948 0.25% 0.013µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13236641.315op/s 13343538.682op/s ± 33466.679op/s 13350135.930op/s ± 17178.862op/s 13364742.473op/s 13382197.529op/s 13414510.584op/s 13428102.333op/s 0.58% -0.744 0.924 0.25% 2366.452op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.892µs 3.912µs ± 0.008µs 3.911µs ± 0.002µs 3.912µs 3.915µs 3.919µs 3.995µs 2.15% 8.450 78.928 0.21% 0.001µs 1 200
credit_card/is_card_number/37828224631 throughput 250300247.726op/s 255651953.776op/s ± 530242.699op/s 255687726.995op/s ± 98397.687op/s 255810401.374op/s 255925840.480op/s 255968666.250op/s 256909736.568op/s 0.48% -8.389 78.191 0.21% 37493.821op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 71.401µs 72.049µs ± 0.176µs 72.061µs ± 0.101µs 72.155µs 72.334µs 72.512µs 72.535µs 0.66% -0.114 1.237 0.24% 0.012µs 1 200
credit_card/is_card_number/378282246310005 throughput 13786380.920op/s 13879493.847op/s ± 33945.337op/s 13877096.561op/s ± 19441.065op/s 13898224.851op/s 13931310.033op/s 13946187.139op/s 14005499.224op/s 0.93% 0.137 1.261 0.24% 2400.298op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 52.121µs 52.206µs ± 0.113µs 52.194µs ± 0.025µs 52.219µs 52.280µs 52.330µs 53.351µs 2.22% 8.268 75.993 0.22% 0.008µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 18743776.480op/s 19154834.495op/s ± 40704.655op/s 19159234.896op/s ± 9066.507op/s 19168266.935op/s 19179166.445op/s 19182822.071op/s 19185990.390op/s 0.14% -8.205 75.154 0.21% 2878.254op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 5.690µs 5.830µs ± 0.076µs 5.829µs ± 0.065µs 5.893µs 5.957µs 5.973µs 5.997µs 2.90% -0.030 -0.869 1.30% 0.005µs 1 200
credit_card/is_card_number/x371413321323331 throughput 166736823.691op/s 171554449.714op/s ± 2239126.609op/s 171566200.947op/s ± 1886342.528op/s 173508255.709op/s 175514505.101op/s 175699825.007op/s 175748334.970op/s 2.44% 0.074 -0.872 1.30% 158330.161op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.890µs 3.910µs ± 0.002µs 3.910µs ± 0.001µs 3.912µs 3.913µs 3.915µs 3.916µs 0.14% -3.097 26.783 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255373883.139op/s 255736082.669op/s ± 152286.446op/s 255728127.612op/s ± 86377.070op/s 255822960.127op/s 255912714.264op/s 255962312.618op/s 257062407.778op/s 0.52% 3.132 27.149 0.06% 10768.278op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 64.102µs 64.304µs ± 0.092µs 64.288µs ± 0.050µs 64.346µs 64.433µs 64.697µs 64.813µs 0.82% 1.975 7.409 0.14% 0.006µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15429046.168op/s 15551100.446op/s ± 22166.798op/s 15555071.510op/s ± 12096.656op/s 15564869.704op/s 15577852.267op/s 15581532.247op/s 15600118.561op/s 0.29% -1.952 7.270 0.14% 1567.429op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 57.375µs 57.558µs ± 0.123µs 57.530µs ± 0.068µs 57.608µs 57.808µs 57.927µs 58.148µs 1.07% 1.623 3.772 0.21% 0.009µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 17197400.860op/s 17373742.509op/s ± 37000.260op/s 17382195.417op/s ± 20719.571op/s 17398938.665op/s 17417094.879op/s 17422948.968op/s 17429199.137op/s 0.27% -1.603 3.669 0.21% 2616.313op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.892µs 3.912µs ± 0.003µs 3.912µs ± 0.002µs 3.914µs 3.917µs 3.921µs 3.921µs 0.23% -0.570 7.053 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255035658.133op/s 255616271.567op/s ± 209874.726op/s 255631883.693op/s ± 117051.021op/s 255749010.043op/s 255866283.440op/s 255953280.228op/s 256930416.755op/s 0.51% 0.591 7.176 0.08% 14840.384op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.557µs 54.801µs ± 0.141µs 54.774µs ± 0.079µs 54.859µs 55.075µs 55.202µs 55.693µs 1.68% 1.842 7.624 0.26% 0.010µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17955442.245op/s 18248114.068op/s ± 46639.294op/s 18256777.992op/s ± 26440.756op/s 18277533.352op/s 18306618.028op/s 18322300.939op/s 18329517.856op/s 0.40% -1.795 7.274 0.25% 3297.896op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 52.120µs 52.185µs ± 0.035µs 52.185µs ± 0.021µs 52.203µs 52.248µs 52.281µs 52.357µs 0.33% 0.983 2.495 0.07% 0.002µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19099810.036op/s 19162536.617op/s ± 12926.053op/s 19162726.581op/s ± 7583.215op/s 19170928.724op/s 19182536.603op/s 19185528.356op/s 19186521.469op/s 0.12% -0.976 2.466 0.07% 914.010op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 5.694µs 5.852µs ± 0.073µs 5.856µs ± 0.047µs 5.900µs 5.969µs 6.004µs 6.038µs 3.10% -0.018 -0.532 1.24% 0.005µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 165625073.291op/s 170909280.983op/s ± 2119691.267op/s 170755021.253op/s ± 1376305.554op/s 172374044.675op/s 174576126.380op/s 175338649.376op/s 175613826.326op/s 2.85% 0.072 -0.542 1.24% 149884.807op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.912µs; 3.913µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/ throughput [255585768.128op/s; 255634231.784op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [80.534µs; 80.744µs] or [-0.130%; +0.130%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12385892.843op/s; 12418091.768op/s] or [-0.130%; +0.130%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [74.917µs; 74.969µs] or [-0.035%; +0.035%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13338900.523op/s; 13348176.842op/s] or [-0.035%; +0.035%] None None None
credit_card/is_card_number/37828224631 execution_time [3.910µs; 3.913µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/37828224631 throughput [255578467.237op/s; 255725440.314op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/378282246310005 execution_time [72.025µs; 72.074µs] or [-0.034%; +0.034%] None None None
credit_card/is_card_number/378282246310005 throughput [13874789.350op/s; 13884198.344op/s] or [-0.034%; +0.034%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.191µs; 52.222µs] or [-0.030%; +0.030%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19149193.221op/s; 19160475.768op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/x371413321323331 execution_time [5.820µs; 5.841µs] or [-0.181%; +0.181%] None None None
credit_card/is_card_number/x371413321323331 throughput [171244128.301op/s; 171864771.127op/s] or [-0.181%; +0.181%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.910µs; 3.911µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/ throughput [255714977.233op/s; 255757188.106op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [64.292µs; 64.317µs] or [-0.020%; +0.020%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15548028.341op/s; 15554172.551op/s] or [-0.020%; +0.020%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [57.541µs; 57.575µs] or [-0.030%; +0.030%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17368614.629op/s; 17378870.390op/s] or [-0.030%; +0.030%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.912µs; 3.913µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255587184.948op/s; 255645358.185op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.781µs; 54.820µs] or [-0.036%; +0.036%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18241650.311op/s; 18254577.826op/s] or [-0.035%; +0.035%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.180µs; 52.190µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19160745.190op/s; 19164328.043op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [5.842µs; 5.862µs] or [-0.172%; +0.172%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [170615512.160op/s; 171203049.807op/s] or [-0.172%; +0.172%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 33.410µs 33.988µs ± 0.993µs 33.535µs ± 0.052µs 33.658µs 36.038µs 36.071µs 38.633µs 15.20% 1.901 2.476 2.91% 0.070µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.850µs; 34.125µs] or [-0.405%; +0.405%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.257µs 26.446µs ± 9.876µs 18.856µs ± 0.427µs 35.521µs 44.921µs 47.889µs 70.089µs 271.70% 0.937 0.449 37.25% 0.698µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [25.077µs; 27.814µs] or [-5.176%; +5.176%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 13.704ms 13.749ms ± 0.030ms 13.745ms ± 0.012ms 13.756ms 13.803ms 13.859ms 13.958ms 1.55% 2.758 12.514 0.22% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [13.745ms; 13.753ms] or [-0.031%; +0.031%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 143.505µs 144.198µs ± 0.295µs 144.151µs ± 0.098µs 144.283µs 144.660µs 145.147µs 146.625µs 1.72% 3.340 22.985 0.20% 0.021µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [144.157µs; 144.239µs] or [-0.028%; +0.028%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 59.843ms 60.271ms ± 2.281ms 59.959ms ± 0.048ms 60.029ms 60.275ms 74.664ms 81.010ms 35.11% 8.200 66.566 3.77% 0.161ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [59.955ms; 60.587ms] or [-0.524%; +0.524%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 08805b5 1753294520 leiyks-add-span-conversion-functions
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 246.335ns 258.598ns ± 14.108ns 251.966ns ± 4.099ns 262.310ns 287.311ns 298.013ns 300.740ns 19.36% 1.332 0.601 5.44% 0.998ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [256.643ns; 260.553ns] or [-0.756%; +0.756%] None None None

Baseline

Omitted due to size.

pr-commenter[bot] avatar Apr 02 '25 15:04 pr-commenter[bot]

Codecov Report

Attention: Patch coverage is 51.76991% with 327 lines in your changes missing coverage. Please review.

Project coverage is 71.02%. Comparing base (4eb2b86) to head (08805b5). Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #994      +/-   ##
==========================================
- Coverage   71.30%   71.02%   -0.29%     
==========================================
  Files         345      348       +3     
  Lines       53148    53839     +691     
==========================================
+ Hits        37898    38239     +341     
- Misses      15250    15600     +350     
Components Coverage Δ
datadog-crashtracker 43.89% <ø> (-0.06%) :arrow_down:
datadog-crashtracker-ffi 5.93% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 86.82% <100.00%> (-0.18%) :arrow_down:
data-pipeline-ffi 88.36% <ø> (ø)
ddcommon 82.99% <100.00%> (+0.10%) :arrow_up:
ddcommon-ffi 70.01% <100.00%> (+0.35%) :arrow_up:
ddtelemetry 60.08% <ø> (ø)
ddtelemetry-ffi 21.32% <ø> (ø)
dogstatsd-client 83.26% <ø> (ø)
datadog-ipc 82.58% <ø> (ø)
datadog-profiling 77.13% <ø> (ø)
datadog-profiling-ffi 62.12% <ø> (ø)
datadog-sidecar 39.10% <23.97%> (-1.14%) :arrow_down:
datdog-sidecar-ffi 11.55% <23.97%> (+6.63%) :arrow_up:
spawn-worker 55.35% <ø> (ø)
tinybytes 91.59% <100.00%> (+0.63%) :arrow_up:
datadog-trace-normalization 98.24% <ø> (ø)
datadog-trace-obfuscation 94.17% <ø> (ø)
datadog-trace-protobuf 77.10% <ø> (ø)
datadog-trace-utils 89.94% <94.82%> (+0.21%) :arrow_up:
datadog-tracer-flare 75.14% <ø> (ø)
datadog-log 76.31% <ø> (ø)
:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov-commenter avatar Apr 02 '25 15:04 codecov-commenter

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 9.24 MB 9.24 MB --.06% (-5.77 KB) 💪
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 22.12 MB 22.05 MB --.32% (-73.65 KB) 💪
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 70.32 MB 70.18 MB --.20% (-147.66 KB) 💪
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 9.16 MB 9.09 MB --.73% (-69.42 KB) 💪
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 26.18 MB 26.10 MB --.33% (-88.62 KB) 💪
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 82.55 MB 82.36 MB --.22% (-192.43 KB) 💪
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 18.40 MB 18.31 MB --.50% (-95.00 KB) 💪
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 65.03 KB 65.03 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 124.94 MB 124.47 MB --.38% (-488.00 KB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 646.92 MB 645.73 MB --.18% (-1.19 MB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 5.89 MB 5.87 MB --.31% (-19.00 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 65.03 KB 65.03 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 17.35 MB 17.31 MB --.22% (-40.00 KB) 💪
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 32.19 MB 32.12 MB --.20% (-67.91 KB) 💪
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 15.69 MB 15.60 MB --.56% (-90.50 KB) 💪
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 66.03 KB 66.03 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 127.33 MB 126.81 MB --.41% (-536.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 636.65 MB 635.45 MB --.18% (-1.19 MB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 4.49 MB 4.48 MB --.24% (-11.50 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 66.03 KB 66.03 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 18.48 MB 18.44 MB --.21% (-40.00 KB) 💪
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 30.23 MB 30.17 MB --.21% (-66.79 KB) 💪
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 63.04 MB 62.91 MB --.20% (-134.00 KB) 💪
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 9.85 MB 9.83 MB --.20% (-20.95 KB) 💪
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so.debug 20.97 MB 20.90 MB --.32% (-69.09 KB) 💪
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 77.49 MB 77.33 MB --.21% (-167.50 KB) 💪
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 9.75 MB 9.74 MB --.16% (-16.66 KB) 💪
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so.debug 24.10 MB 24.02 MB --.32% (-80.21 KB) 💪

r1viollet avatar Apr 04 '25 15:04 r1viollet

The Data Pipeline team hasn't seen any design docs or had any conversations about adding this functionality. Also, I'm not ok with tests that can only run on nightly. Our CI is on stable, and we build our releases with stable.

It's an error on my part, the PR was supposed to stay in draft as it is just at the experiment level yet. Sorry for the inconvenience :bow:

Leiyks avatar Apr 08 '25 10:04 Leiyks

Assuming the code works, it's now time to get the CI checks here green :-) Looks good so far.

bwoebi avatar Apr 30 '25 19:04 bwoebi

We try to keep data-pipeline-ffi as a thin/optional layer over data-pipeline functionalities which are used directly by some crates (e.g. dd-trace-py, dd-trace-rs)

The Span struct and it's logic lives in trace-utils and is not specific to data-pipeline

+1, it's important we maintain parity between the FFI layer and our Rust API for Data Pipeline. trace-utils sounds like a natural home for this. We can also migrate it to Data Pipeline later.

ekump avatar May 05 '25 16:05 ekump

@ekump So the suggestion is having datadog-trace-utils-ffi, right?

bwoebi avatar May 05 '25 17:05 bwoebi

@ekump So the suggestion is having datadog-trace-utils-ffi, right?

My strong opinion is that the real logic doesn't live in an FFI crate. I think it should live in datadog-trace-utils (for now, at least). And whatever FFI crate we use should call datadog-trace-utils.

I'm hesitant to suggest datadog-trace-utils-ffi because we want to rethink datadog-trace-utils in general and potentially break it up. (Something @paullegranddc has been thinking about). Is this something that could live in datadog-sidecar-ffi for now?

ekump avatar May 06 '25 20:05 ekump

Alright, let's put it in datadog-sidecar-ffi into a separate file for now, as long as we don't know the ultimate fate of this. But the code in that file is really just ffi glue code.

bwoebi avatar May 07 '25 00:05 bwoebi

No data reported at this time.
This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 08805b5 | Was this helpful? Give us feedback!

/merge

bwoebi avatar Jul 23 '25 15:07 bwoebi

View all feedbacks in Devflow UI.

2025-07-23 15:51:41 UTC :information_source: Start processing command /merge


2025-07-23 15:51:55 UTC :information_source: MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals. Note: if you pushed new commits since the last approval, you may need additional approval. You can remove it from the waiting list with /remove command.


2025-07-23 18:52:14 UTC :information_source: MergeQueue: merge request added to the queue

The expected merge time in main is approximately 39m (p90).


2025-07-23 19:28:15 UTC :information_source: MergeQueue: This merge request was merged