libdatadog icon indicating copy to clipboard operation
libdatadog copied to clipboard

[WIP] Removing serde from deserialization

Open hoolioh opened this issue 1 year ago • 2 comments
trafficstars

What does this PR do?

This PR replaces serde for rmp in order to deserialize TracerPayloads.

Motivation

The current method for deserializing is taces is too memory intensive so we need a low level deserialization process to start peeling off the inefiencies by reducing allocations.

Additional Notes

We chose to use rmp exclusively in order to build the solution from there. Probably next steps will involve to use rmpv or antoher high level crates to handle the msgpack payload if we can achieve the same level of performance as we do with rmp.

hoolioh avatar Jul 18 '24 16:07 hoolioh

Benchmarks

This comment was omitted because it was over 65536 characters.Please check the Gitlab Job logs to see its output.

pr-commenter[bot] avatar Jul 18 '24 16:07 pr-commenter[bot]

Codecov Report

Attention: Patch coverage is 76.11940% with 160 lines in your changes missing coverage. Please review.

Project coverage is 70.71%. Comparing base (d351d66) to head (aff277e). Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #537      +/-   ##
==========================================
+ Coverage   70.58%   70.71%   +0.12%     
==========================================
  Files         206      211       +5     
  Lines       28129    28797     +668     
==========================================
+ Hits        19856    20364     +508     
- Misses       8273     8433     +160     
Components Coverage Δ
crashtracker 25.46% <70.25%> (+8.94%) :arrow_up:
datadog-alloc 98.73% <ø> (ø)
data-pipeline 50.00% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 86.43% <ø> (ø)
ddcommon-ffi 75.29% <ø> (ø)
ddtelemetry 58.95% <ø> (ø)
ipc 84.13% <ø> (ø)
profiling 78.09% <0.00%> (-0.69%) :arrow_down:
profiling-ffi 56.76% <0.00%> (-1.88%) :arrow_down:
serverless 0.00% <ø> (ø)
sidecar 35.42% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 54.98% <ø> (ø)
trace-mini-agent 71.14% <ø> (ø)
trace-normalization 98.24% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.16% <ø> (ø)
trace-utils 90.91% <90.12%> (-0.13%) :arrow_down:

codecov-commenter avatar Jul 19 '24 09:07 codecov-commenter