feat(llmobs): add prompt tracking telemetry tags
Description
Adds prompt tracking telemetry tags for OpenAI integration:
- Adds
prompt_tracking_instrumentation_method:autotag for auto-instrumented prompts (OpenAI, LangChain) - Adds
prompt_multimodal:truetag for OpenAI prompts containing image/file inputs
Motivation
Enable dd-go to emit statsd metrics tagged by prompt tracking source and multimodal content for internal dashboard analytics.
Testing
- Updated unit tests for OpenAI and LangChain integrations
- All existing tests pass with new tag assertions
- System tests
Risks
๐คท
Related PRs
- dd-trace-js: https://github.com/DataDog/dd-trace-js/pull/7106
- dd-source: https://github.com/DataDog/dd-source/pull/323345
- system-tests: https://github.com/DataDog/system-tests/pull/5876
CODEOWNERS have been resolved as:
ddtrace/llmobs/_constants.py @DataDog/ml-observability
ddtrace/llmobs/_integrations/langchain.py @DataDog/ml-observability
ddtrace/llmobs/_integrations/utils.py @DataDog/ml-observability
ddtrace/llmobs/_llmobs.py @DataDog/ml-observability
tests/contrib/langchain/test_langchain_llmobs.py @DataDog/ml-observability
tests/contrib/openai/test_openai_llmobs.py @DataDog/ml-observability
tests/contrib/openai/utils.py @DataDog/ml-observability
tests/llmobs/_utils.py @DataDog/ml-observability
tests/llmobs/test_llmobs_service.py @DataDog/ml-observability
Bootstrap import analysis
Comparison of import times between this PR and base.
Summary
The average import time from this PR is: 267 ยฑ 7 ms.
The average import time from base is: 269 ยฑ 7 ms.
The import time difference between this PR and base is: -2.7 ยฑ 0.3 ms.
Import time breakdown
The following import paths have shrunk:
ddtrace.auto
2.726 ms
(1.02%)
ddtrace
1.380 ms
(0.52%)
ddtrace._logger
0.682 ms
(0.26%)
ddtrace.internal.telemetry
0.682 ms
(0.26%)
ddtrace.internal.telemetry.writer
0.682 ms
(0.26%)
ddtrace.internal.utils.version
0.682 ms
(0.26%)
ddtrace.version
0.682 ms
(0.26%)
ddtrace.internal._unpatched
0.034 ms
(0.01%)
json
0.034 ms
(0.01%)
json.decoder
0.034 ms
(0.01%)
re
0.034 ms
(0.01%)
enum
0.034 ms
(0.01%)
types
0.034 ms
(0.01%)
ddtrace.bootstrap.sitecustomize
1.346 ms
(0.50%)
ddtrace.bootstrap.preload
1.346 ms
(0.50%)
ddtrace.internal.remoteconfig.client
0.634 ms
(0.24%)
Performance SLOs
Comparing candidate alex/MLOB-4895_prompt-tracking-auto-marker (ccee92bf) with baseline main (09ca1fb2)
๐ Performance Regressions (3 suites)
๐ iastaspects - 118/118
โ add_aspect
Time: โ 18.192ยตs (SLO: <20.000ยตs -9.0%) vs baseline: ๐ +21.2%
Memory: โ 42.959MB (SLO: <43.250MB ๐ก -0.7%) vs baseline: +4.9%
โ add_inplace_aspect
Time: โ 15.121ยตs (SLO: <20.000ยตs ๐ -24.4%) vs baseline: -0.2%
Memory: โ 42.998MB (SLO: <43.250MB ๐ก -0.6%) vs baseline: +4.9%
โ add_inplace_noaspect
Time: โ 0.336ยตs (SLO: <10.000ยตs ๐ -96.6%) vs baseline: -0.3%
Memory: โ 42.979MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.7%
โ add_noaspect
Time: โ 0.544ยตs (SLO: <10.000ยตs ๐ -94.6%) vs baseline: -2.0%
Memory: โ 43.057MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.1%
โ bytearray_aspect
Time: โ 17.935ยตs (SLO: <30.000ยตs ๐ -40.2%) vs baseline: -1.6%
Memory: โ 42.998MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.8%
โ bytearray_extend_aspect
Time: โ 23.781ยตs (SLO: <30.000ยตs ๐ -20.7%) vs baseline: -0.4%
Memory: โ 43.136MB (SLO: <43.500MB ๐ก -0.8%) vs baseline: +5.2%
โ bytearray_extend_noaspect
Time: โ 2.627ยตs (SLO: <10.000ยตs ๐ -73.7%) vs baseline: -1.5%
Memory: โ 42.998MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.7%
โ bytearray_noaspect
Time: โ 1.455ยตs (SLO: <10.000ยตs ๐ -85.5%) vs baseline: -0.3%
Memory: โ 43.077MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.0%
โ bytes_aspect
Time: โ 16.808ยตs (SLO: <20.000ยตs ๐ -16.0%) vs baseline: -0.5%
Memory: โ 43.018MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +4.8%
โ bytes_noaspect
Time: โ 1.397ยตs (SLO: <10.000ยตs ๐ -86.0%) vs baseline: +0.8%
Memory: โ 43.077MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.1%
โ bytesio_aspect
Time: โ 55.261ยตs (SLO: <70.000ยตs ๐ -21.1%) vs baseline: -0.9%
Memory: โ 42.979MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.8%
โ bytesio_noaspect
Time: โ 3.211ยตs (SLO: <10.000ยตs ๐ -67.9%) vs baseline: -0.3%
Memory: โ 43.077MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.1%
โ capitalize_aspect
Time: โ 14.681ยตs (SLO: <20.000ยตs ๐ -26.6%) vs baseline: -1.0%
Memory: โ 43.037MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +5.0%
โ capitalize_noaspect
Time: โ 2.585ยตs (SLO: <10.000ยตs ๐ -74.2%) vs baseline: +1.0%
Memory: โ 42.979MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.7%
โ casefold_aspect
Time: โ 14.732ยตs (SLO: <20.000ยตs ๐ -26.3%) vs baseline: -0.9%
Memory: โ 43.018MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +5.0%
โ casefold_noaspect
Time: โ 3.071ยตs (SLO: <10.000ยตs ๐ -69.3%) vs baseline: -0.5%
Memory: โ 43.018MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +5.0%
โ decode_aspect
Time: โ 15.576ยตs (SLO: <30.000ยตs ๐ -48.1%) vs baseline: -0.2%
Memory: โ 42.939MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.6%
โ decode_noaspect
Time: โ 1.591ยตs (SLO: <10.000ยตs ๐ -84.1%) vs baseline: ~same
Memory: โ 43.037MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +4.8%
โ encode_aspect
Time: โ 18.334ยตs (SLO: <30.000ยตs ๐ -38.9%) vs baseline: ๐ +22.0%
Memory: โ 42.979MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.6%
โ encode_noaspect
Time: โ 1.465ยตs (SLO: <10.000ยตs ๐ -85.3%) vs baseline: -1.2%
Memory: โ 42.979MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.8%
โ format_aspect
Time: โ 171.185ยตs (SLO: <200.000ยตs ๐ -14.4%) vs baseline: ~same
Memory: โ 43.175MB (SLO: <43.250MB ๐ก -0.2%) vs baseline: +5.0%
โ format_map_aspect
Time: โ 191.786ยตs (SLO: <200.000ยตs -4.1%) vs baseline: +0.1%
Memory: โ 43.077MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +4.5%
โ format_map_noaspect
Time: โ 3.759ยตs (SLO: <10.000ยตs ๐ -62.4%) vs baseline: +0.7%
Memory: โ 43.037MB (SLO: <43.250MB ๐ก -0.5%) vs baseline: +5.1%
โ format_noaspect
Time: โ 3.164ยตs (SLO: <10.000ยตs ๐ -68.4%) vs baseline: -0.1%
Memory: โ 42.959MB (SLO: <43.250MB ๐ก -0.7%) vs baseline: +4.7%
โ index_aspect
Time: โ 15.326ยตs (SLO: <20.000ยตs ๐ -23.4%) vs baseline: -0.2%
Memory: โ 42.939MB (SLO: <43.250MB ๐ก -0.7%) vs baseline: +4.8%
โ index_noaspect
Time: โ 0.462ยตs (SLO: <10.000ยตs ๐ -95.4%) vs baseline: ~same
Memory: โ 42.959MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.5%
โ join_aspect
Time: โ 16.946ยตs (SLO: <20.000ยตs ๐ -15.3%) vs baseline: -0.5%
Memory: โ 42.998MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.8%
โ join_noaspect
Time: โ 1.485ยตs (SLO: <10.000ยตs ๐ -85.2%) vs baseline: -1.4%
Memory: โ 43.057MB (SLO: <43.250MB ๐ก -0.4%) vs baseline: +5.0%
โ ljust_aspect
Time: โ 21.002ยตs (SLO: <30.000ยตs ๐ -30.0%) vs baseline: +0.2%
Memory: โ 43.037MB (SLO: <43.250MB ๐ก -0.5%) vs baseline: +4.9%
โ ljust_noaspect
Time: โ 2.701ยตs (SLO: <10.000ยตs ๐ -73.0%) vs baseline: +0.7%
Memory: โ 42.979MB (SLO: <43.250MB ๐ก -0.6%) vs baseline: +4.9%
โ lower_aspect
Time: โ 17.933ยตs (SLO: <30.000ยตs ๐ -40.2%) vs baseline: -0.9%
Memory: โ 42.939MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.6%
โ lower_noaspect
Time: โ 2.375ยตs (SLO: <10.000ยตs ๐ -76.3%) vs baseline: -0.3%
Memory: โ 42.979MB (SLO: <43.250MB ๐ก -0.6%) vs baseline: +4.7%
โ lstrip_aspect
Time: โ 17.635ยตs (SLO: <20.000ยตs ๐ -11.8%) vs baseline: ~same
Memory: โ 42.998MB (SLO: <43.250MB ๐ก -0.6%) vs baseline: +4.9%
โ lstrip_noaspect
Time: โ 1.821ยตs (SLO: <10.000ยตs ๐ -81.8%) vs baseline: -0.1%
Memory: โ 42.920MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.8%
โ modulo_aspect
Time: โ 166.374ยตs (SLO: <200.000ยตs ๐ -16.8%) vs baseline: -0.4%
Memory: โ 43.037MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +4.7%
โ modulo_aspect_for_bytearray_bytearray
Time: โ 174.271ยตs (SLO: <200.000ยตs ๐ -12.9%) vs baseline: -0.1%
Memory: โ 43.077MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.0%
โ modulo_aspect_for_bytes
Time: โ 168.042ยตs (SLO: <200.000ยตs ๐ -16.0%) vs baseline: -0.5%
Memory: โ 43.018MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +4.7%
โ modulo_aspect_for_bytes_bytearray
Time: โ 171.796ยตs (SLO: <200.000ยตs ๐ -14.1%) vs baseline: -0.3%
Memory: โ 43.096MB (SLO: <43.500MB ๐ก -0.9%) vs baseline: +4.8%
โ modulo_noaspect
Time: โ 3.690ยตs (SLO: <10.000ยตs ๐ -63.1%) vs baseline: -0.9%
Memory: โ 43.037MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +4.9%
โ replace_aspect
Time: โ 214.413ยตs (SLO: <300.000ยตs ๐ -28.5%) vs baseline: -0.2%
Memory: โ 43.155MB (SLO: <44.000MB ๐ก -1.9%) vs baseline: +5.2%
โ replace_noaspect
Time: โ 5.200ยตs (SLO: <10.000ยตs ๐ -48.0%) vs baseline: ~same
Memory: โ 43.077MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.1%
โ repr_aspect
Time: โ 1.394ยตs (SLO: <10.000ยตs ๐ -86.1%) vs baseline: +0.4%
Memory: โ 42.841MB (SLO: <43.500MB ๐ก -1.5%) vs baseline: +4.4%
โ repr_noaspect
Time: โ 0.525ยตs (SLO: <10.000ยตs ๐ -94.8%) vs baseline: -0.3%
Memory: โ 42.998MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +5.1%
โ rstrip_aspect
Time: โ 23.062ยตs (SLO: <30.000ยตs ๐ -23.1%) vs baseline: ๐ +19.2%
Memory: โ 43.057MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.0%
โ rstrip_noaspect
Time: โ 1.885ยตs (SLO: <10.000ยตs ๐ -81.2%) vs baseline: +1.2%
Memory: โ 42.939MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.4%
โ slice_aspect
Time: โ 16.003ยตs (SLO: <20.000ยตs ๐ -20.0%) vs baseline: -0.4%
Memory: โ 42.939MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.6%
โ slice_noaspect
Time: โ 0.597ยตs (SLO: <10.000ยตs ๐ -94.0%) vs baseline: -0.5%
Memory: โ 42.979MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.7%
โ stringio_aspect
Time: โ 54.330ยตs (SLO: <80.000ยตs ๐ -32.1%) vs baseline: +0.1%
Memory: โ 43.057MB (SLO: <43.500MB ๐ก -1.0%) vs baseline: +5.0%
โ stringio_noaspect
Time: โ 3.542ยตs (SLO: <10.000ยตs ๐ -64.6%) vs baseline: +0.7%
Memory: โ 42.920MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.6%
โ strip_aspect
Time: โ 17.648ยตs (SLO: <20.000ยตs ๐ -11.8%) vs baseline: -1.1%
Memory: โ 42.998MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.7%
โ strip_noaspect
Time: โ 1.821ยตs (SLO: <10.000ยตs ๐ -81.8%) vs baseline: -0.8%
Memory: โ 43.018MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +4.8%
โ swapcase_aspect
Time: โ 18.532ยตs (SLO: <30.000ยตs ๐ -38.2%) vs baseline: -0.4%
Memory: โ 43.018MB (SLO: <43.500MB ๐ก -1.1%) vs baseline: +5.0%
โ swapcase_noaspect
Time: โ 2.737ยตs (SLO: <10.000ยตs ๐ -72.6%) vs baseline: -0.7%
Memory: โ 42.979MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.8%
โ title_aspect
Time: โ 18.293ยตs (SLO: <20.000ยตs -8.5%) vs baseline: -0.4%
Memory: โ 42.959MB (SLO: <43.000MB ๐ก ~same) vs baseline: +4.8%
โ title_noaspect
Time: โ 2.656ยตs (SLO: <10.000ยตs ๐ -73.4%) vs baseline: +0.2%
Memory: โ 42.920MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.6%
โ translate_aspect
Time: โ 20.566ยตs (SLO: <30.000ยตs ๐ -31.4%) vs baseline: -0.4%
Memory: โ 42.959MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.8%
โ translate_noaspect
Time: โ 4.260ยตs (SLO: <10.000ยตs ๐ -57.4%) vs baseline: ~same
Memory: โ 42.939MB (SLO: <43.500MB ๐ก -1.3%) vs baseline: +4.5%
โ upper_aspect
Time: โ 17.973ยตs (SLO: <30.000ยตs ๐ -40.1%) vs baseline: -1.1%
Memory: โ 42.959MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +4.8%
โ upper_noaspect
Time: โ 2.354ยตs (SLO: <10.000ยตs ๐ -76.5%) vs baseline: -1.7%
Memory: โ 42.998MB (SLO: <43.500MB ๐ก -1.2%) vs baseline: +5.0%
๐ iastaspectsospath - 24/24
โ ospathbasename_aspect
Time: โ 5.141ยตs (SLO: <10.000ยตs ๐ -48.6%) vs baseline: ๐ +21.9%
Memory: โ 41.366MB (SLO: <43.500MB -4.9%) vs baseline: +4.5%
โ ospathbasename_noaspect
Time: โ 4.276ยตs (SLO: <10.000ยตs ๐ -57.2%) vs baseline: -1.0%
Memory: โ 41.484MB (SLO: <43.500MB -4.6%) vs baseline: +5.0%
โ ospathjoin_aspect
Time: โ 6.154ยตs (SLO: <10.000ยตs ๐ -38.5%) vs baseline: +0.3%
Memory: โ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +5.1%
โ ospathjoin_noaspect
Time: โ 6.167ยตs (SLO: <10.000ยตs ๐ -38.3%) vs baseline: -1.0%
Memory: โ 41.524MB (SLO: <43.500MB -4.5%) vs baseline: +4.9%
โ ospathnormcase_aspect
Time: โ 3.514ยตs (SLO: <10.000ยตs ๐ -64.9%) vs baseline: +0.5%
Memory: โ 41.504MB (SLO: <43.500MB -4.6%) vs baseline: +5.1%
โ ospathnormcase_noaspect
Time: โ 3.561ยตs (SLO: <10.000ยตs ๐ -64.4%) vs baseline: -0.7%
Memory: โ 41.347MB (SLO: <43.500MB -5.0%) vs baseline: +4.7%
โ ospathsplit_aspect
Time: โ 4.809ยตs (SLO: <10.000ยตs ๐ -51.9%) vs baseline: -0.4%
Memory: โ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +5.0%
โ ospathsplit_noaspect
Time: โ 4.962ยตs (SLO: <10.000ยตs ๐ -50.4%) vs baseline: +0.5%
Memory: โ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9%
โ ospathsplitdrive_aspect
Time: โ 3.801ยตs (SLO: <10.000ยตs ๐ -62.0%) vs baseline: +0.6%
Memory: โ 41.425MB (SLO: <43.500MB -4.8%) vs baseline: +4.9%
โ ospathsplitdrive_noaspect
Time: โ 0.753ยตs (SLO: <10.000ยตs ๐ -92.5%) vs baseline: +1.0%
Memory: โ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +4.8%
โ ospathsplitext_aspect
Time: โ 4.659ยตs (SLO: <10.000ยตs ๐ -53.4%) vs baseline: +0.6%
Memory: โ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +5.1%
โ ospathsplitext_noaspect
Time: โ 4.712ยตs (SLO: <10.000ยตs ๐ -52.9%) vs baseline: +1.6%
Memory: โ 41.465MB (SLO: <43.500MB -4.7%) vs baseline: +5.0%
๐ telemetryaddmetric - 30/30
โ 1-count-metric-1-times
Time: โ 3.359ยตs (SLO: <20.000ยตs ๐ -83.2%) vs baseline: ๐ +10.6%
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.9%
โ 1-count-metrics-100-times
Time: โ 202.966ยตs (SLO: <220.000ยตs -7.7%) vs baseline: +1.4%
Memory: โ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +4.7%
โ 1-distribution-metric-1-times
Time: โ 3.364ยตs (SLO: <20.000ยตs ๐ -83.2%) vs baseline: -0.1%
Memory: โ 34.800MB (SLO: <35.500MB ๐ก -2.0%) vs baseline: +4.9%
โ 1-distribution-metrics-100-times
Time: โ 216.828ยตs (SLO: <230.000ยตs -5.7%) vs baseline: +0.3%
Memory: โ 34.918MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.3%
โ 1-gauge-metric-1-times
Time: โ 2.199ยตs (SLO: <20.000ยตs ๐ -89.0%) vs baseline: ~same
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.8%
โ 1-gauge-metrics-100-times
Time: โ 137.345ยตs (SLO: <150.000ยตs -8.4%) vs baseline: +0.6%
Memory: โ 34.859MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.9%
โ 1-rate-metric-1-times
Time: โ 3.163ยตs (SLO: <20.000ยตs ๐ -84.2%) vs baseline: +1.1%
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +5.3%
โ 1-rate-metrics-100-times
Time: โ 214.501ยตs (SLO: <250.000ยตs ๐ -14.2%) vs baseline: ~same
Memory: โ 34.819MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.6%
โ 100-count-metrics-100-times
Time: โ 20.189ms (SLO: <22.000ms -8.2%) vs baseline: ~same
Memory: โ 34.859MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.9%
โ 100-distribution-metrics-100-times
Time: โ 2.309ms (SLO: <2.550ms -9.4%) vs baseline: +0.3%
Memory: โ 34.839MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.5%
โ 100-gauge-metrics-100-times
Time: โ 1.410ms (SLO: <1.550ms -9.1%) vs baseline: +0.6%
Memory: โ 34.898MB (SLO: <35.500MB ๐ก -1.7%) vs baseline: +5.1%
โ 100-rate-metrics-100-times
Time: โ 2.202ms (SLO: <2.550ms ๐ -13.6%) vs baseline: ~same
Memory: โ 34.819MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.7%
โ flush-1-metric
Time: โ 4.656ยตs (SLO: <20.000ยตs ๐ -76.7%) vs baseline: +0.9%
Memory: โ 35.134MB (SLO: <35.500MB ๐ก -1.0%) vs baseline: +4.6%
โ flush-100-metrics
Time: โ 174.009ยตs (SLO: <250.000ยตs ๐ -30.4%) vs baseline: +0.5%
Memory: โ 35.212MB (SLO: <35.500MB ๐ก -0.8%) vs baseline: +4.6%
โ flush-1000-metrics
Time: โ 2.170ms (SLO: <2.500ms ๐ -13.2%) vs baseline: -0.5%
Memory: โ 35.960MB (SLO: <36.500MB ๐ก -1.5%) vs baseline: +4.6%
๐ก Near SLO Breach (15 suites)
๐ก coreapiscenario - 10/10 (1 unstable)
โ ๏ธ context_with_data_listeners
Time: โ ๏ธ 13.288ยตs (SLO: <20.000ยตs ๐ -33.6%) vs baseline: +0.5%
Memory: โ 34.819MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.8%
โ context_with_data_no_listeners
Time: โ 3.280ยตs (SLO: <10.000ยตs ๐ -67.2%) vs baseline: -0.8%
Memory: โ 34.800MB (SLO: <35.500MB ๐ก -2.0%) vs baseline: +5.0%
โ get_item_exists
Time: โ 0.585ยตs (SLO: <10.000ยตs ๐ -94.1%) vs baseline: +0.8%
Memory: โ 34.918MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.4%
โ get_item_missing
Time: โ 0.639ยตs (SLO: <10.000ยตs ๐ -93.6%) vs baseline: +0.4%
Memory: โ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +4.9%
โ set_item
Time: โ 24.454ยตs (SLO: <30.000ยตs ๐ -18.5%) vs baseline: +1.5%
Memory: โ 34.859MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +5.2%
๐ก djangosimple - 30/30
โ appsec
Time: โ 19.569ms (SLO: <22.300ms ๐ -12.2%) vs baseline: ~same
Memory: โ 68.282MB (SLO: <70.500MB -3.1%) vs baseline: +4.8%
โ exception-replay-enabled
Time: โ 1.365ms (SLO: <1.450ms -5.8%) vs baseline: +0.2%
Memory: โ 66.331MB (SLO: <67.500MB ๐ก -1.7%) vs baseline: +4.9%
โ iast
Time: โ 19.630ms (SLO: <22.250ms ๐ -11.8%) vs baseline: -0.3%
Memory: โ 68.233MB (SLO: <70.000MB -2.5%) vs baseline: +4.9%
โ profiler
Time: โ 14.681ms (SLO: <16.550ms ๐ -11.3%) vs baseline: -0.3%
Memory: โ 56.191MB (SLO: <57.500MB -2.3%) vs baseline: +5.0%
โ resource-renaming
Time: โ 19.521ms (SLO: <21.750ms ๐ -10.2%) vs baseline: +0.6%
Memory: โ 68.243MB (SLO: <70.500MB -3.2%) vs baseline: +4.8%
โ span-code-origin
Time: โ 19.921ms (SLO: <28.200ms ๐ -29.4%) vs baseline: +0.7%
Memory: โ 68.165MB (SLO: <71.000MB -4.0%) vs baseline: +4.8%
โ tracer
Time: โ 19.649ms (SLO: <21.750ms -9.7%) vs baseline: +0.6%
Memory: โ 68.282MB (SLO: <70.000MB -2.5%) vs baseline: +4.9%
โ tracer-and-profiler
Time: โ 20.826ms (SLO: <23.500ms ๐ -11.4%) vs baseline: -0.3%
Memory: โ 69.245MB (SLO: <71.000MB -2.5%) vs baseline: +4.9%
โ tracer-dont-create-db-spans
Time: โ 19.591ms (SLO: <21.500ms -8.9%) vs baseline: -0.5%
Memory: โ 68.321MB (SLO: <70.000MB -2.4%) vs baseline: +5.0%
โ tracer-minimal
Time: โ 16.804ms (SLO: <17.500ms -4.0%) vs baseline: ~same
Memory: โ 67.967MB (SLO: <70.000MB -2.9%) vs baseline: +4.8%
โ tracer-native
Time: โ 19.511ms (SLO: <21.750ms ๐ -10.3%) vs baseline: +0.5%
Memory: โ 68.203MB (SLO: <72.500MB -5.9%) vs baseline: +4.8%
โ tracer-no-caches
Time: โ 17.617ms (SLO: <19.650ms ๐ -10.3%) vs baseline: +0.1%
Memory: โ 67.987MB (SLO: <70.000MB -2.9%) vs baseline: +4.6%
โ tracer-no-databases
Time: โ 19.081ms (SLO: <20.100ms -5.1%) vs baseline: -0.5%
Memory: โ 67.869MB (SLO: <70.000MB -3.0%) vs baseline: +4.8%
โ tracer-no-middleware
Time: โ 19.261ms (SLO: <21.500ms ๐ -10.4%) vs baseline: -0.5%
Memory: โ 68.203MB (SLO: <70.000MB -2.6%) vs baseline: +4.8%
โ tracer-no-templates
Time: โ 19.627ms (SLO: <22.000ms ๐ -10.8%) vs baseline: +1.0%
Memory: โ 68.229MB (SLO: <70.500MB -3.2%) vs baseline: +4.9%
๐ก errortrackingdjangosimple - 6/6
โ errortracking-enabled-all
Time: โ 16.358ms (SLO: <19.850ms ๐ -17.6%) vs baseline: +0.2%
Memory: โ 69.745MB (SLO: <70.000MB ๐ก -0.4%) vs baseline: +4.8%
โ errortracking-enabled-user
Time: โ 16.350ms (SLO: <19.400ms ๐ -15.7%) vs baseline: +0.4%
Memory: โ 69.846MB (SLO: <70.000MB ๐ก -0.2%) vs baseline: +4.9%
โ tracer-enabled
Time: โ 16.335ms (SLO: <19.450ms ๐ -16.0%) vs baseline: +0.3%
Memory: โ 69.839MB (SLO: <70.000MB ๐ก -0.2%) vs baseline: +4.9%
๐ก errortrackingflasksqli - 6/6
โ errortracking-enabled-all
Time: โ 2.067ms (SLO: <2.300ms ๐ -10.1%) vs baseline: -0.1%
Memory: โ 55.719MB (SLO: <56.500MB ๐ก -1.4%) vs baseline: +4.7%
โ errortracking-enabled-user
Time: โ 2.077ms (SLO: <2.250ms -7.7%) vs baseline: +0.2%
Memory: โ 55.719MB (SLO: <56.500MB ๐ก -1.4%) vs baseline: +5.0%
โ tracer-enabled
Time: โ 2.064ms (SLO: <2.300ms ๐ -10.3%) vs baseline: -0.2%
Memory: โ 55.679MB (SLO: <56.500MB ๐ก -1.5%) vs baseline: +5.0%
๐ก flasksimple - 18/18
โ appsec-get
Time: โ 3.367ms (SLO: <4.750ms ๐ -29.1%) vs baseline: -0.3%
Memory: โ 55.507MB (SLO: <66.500MB ๐ -16.5%) vs baseline: +4.9%
โ appsec-post
Time: โ 2.855ms (SLO: <6.750ms ๐ -57.7%) vs baseline: ~same
Memory: โ 55.762MB (SLO: <66.500MB ๐ -16.1%) vs baseline: +5.2%
โ appsec-telemetry
Time: โ 3.379ms (SLO: <4.750ms ๐ -28.9%) vs baseline: ~same
Memory: โ 55.415MB (SLO: <66.500MB ๐ -16.7%) vs baseline: +4.9%
โ debugger
Time: โ 1.872ms (SLO: <2.000ms -6.4%) vs baseline: +0.2%
Memory: โ 47.806MB (SLO: <49.500MB -3.4%) vs baseline: +4.8%
โ iast-get
Time: โ 1.852ms (SLO: <2.000ms -7.4%) vs baseline: -0.1%
Memory: โ 44.797MB (SLO: <49.000MB -8.6%) vs baseline: +4.6%
โ profiler
Time: โ 1.861ms (SLO: <2.100ms ๐ -11.4%) vs baseline: -0.1%
Memory: โ 48.767MB (SLO: <50.000MB -2.5%) vs baseline: +5.1%
โ resource-renaming
Time: โ 3.358ms (SLO: <3.650ms -8.0%) vs baseline: -0.3%
Memory: โ 55.497MB (SLO: <56.000MB ๐ก -0.9%) vs baseline: +4.8%
โ tracer
Time: โ 3.374ms (SLO: <3.650ms -7.6%) vs baseline: +0.1%
Memory: โ 55.431MB (SLO: <56.500MB ๐ก -1.9%) vs baseline: +4.5%
โ tracer-native
Time: โ 3.366ms (SLO: <3.650ms -7.8%) vs baseline: +0.1%
Memory: โ 55.625MB (SLO: <60.000MB -7.3%) vs baseline: +5.1%
๐ก flasksqli - 6/6
โ appsec-enabled
Time: โ 2.066ms (SLO: <4.200ms ๐ -50.8%) vs baseline: +0.2%
Memory: โ 55.679MB (SLO: <66.000MB ๐ -15.6%) vs baseline: +4.7%
โ iast-enabled
Time: โ 2.065ms (SLO: <2.800ms ๐ -26.2%) vs baseline: +0.2%
Memory: โ 55.719MB (SLO: <62.500MB ๐ -10.9%) vs baseline: +5.0%
โ tracer-enabled
Time: โ 2.066ms (SLO: <2.250ms -8.2%) vs baseline: +0.4%
Memory: โ 55.679MB (SLO: <56.500MB ๐ก -1.5%) vs baseline: +4.8%
๐ก httppropagationextract - 60/60
โ all_styles_all_headers
Time: โ 81.568ยตs (SLO: <100.000ยตs ๐ -18.4%) vs baseline: +1.1%
Memory: โ 34.957MB (SLO: <35.500MB ๐ก -1.5%) vs baseline: +5.0%
โ b3_headers
Time: โ 14.112ยตs (SLO: <20.000ยตs ๐ -29.4%) vs baseline: ~same
Memory: โ 34.918MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.7%
โ b3_single_headers
Time: โ 13.355ยตs (SLO: <20.000ยตs ๐ -33.2%) vs baseline: +1.3%
Memory: โ 34.957MB (SLO: <35.500MB ๐ก -1.5%) vs baseline: +5.0%
โ datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_match
Time: โ 63.887ยตs (SLO: <80.000ยตs ๐ -20.1%) vs baseline: +0.5%
Memory: โ 34.859MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.8%
โ datadog_tracecontext_tracestate_propagated_on_trace_id_match
Time: โ 66.105ยตs (SLO: <80.000ยตs ๐ -17.4%) vs baseline: +0.6%
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.9%
โ empty_headers
Time: โ 1.625ยตs (SLO: <10.000ยตs ๐ -83.7%) vs baseline: +0.8%
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.4%
โ full_t_id_datadog_headers
Time: โ 22.419ยตs (SLO: <30.000ยตs ๐ -25.3%) vs baseline: +0.2%
Memory: โ 34.996MB (SLO: <35.500MB ๐ก -1.4%) vs baseline: +5.2%
โ invalid_priority_header
Time: โ 6.549ยตs (SLO: <10.000ยตs ๐ -34.5%) vs baseline: -0.4%
Memory: โ 34.898MB (SLO: <35.500MB ๐ก -1.7%) vs baseline: +4.9%
โ invalid_span_id_header
Time: โ 6.578ยตs (SLO: <10.000ยตs ๐ -34.2%) vs baseline: +0.3%
Memory: โ 35.036MB (SLO: <35.500MB ๐ก -1.3%) vs baseline: +5.2%
โ invalid_tags_header
Time: โ 6.542ยตs (SLO: <10.000ยตs ๐ -34.6%) vs baseline: -0.3%
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.2%
โ invalid_trace_id_header
Time: โ 6.564ยตs (SLO: <10.000ยตs ๐ -34.4%) vs baseline: -0.5%
Memory: โ 35.016MB (SLO: <35.500MB ๐ก -1.4%) vs baseline: +5.3%
โ large_header_no_matches
Time: โ 27.967ยตs (SLO: <30.000ยตs -6.8%) vs baseline: +0.7%
Memory: โ 34.859MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.6%
โ large_valid_headers_all
Time: โ 29.082ยตs (SLO: <40.000ยตs ๐ -27.3%) vs baseline: +0.8%
Memory: โ 34.918MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.6%
โ medium_header_no_matches
Time: โ 9.895ยตs (SLO: <20.000ยตs ๐ -50.5%) vs baseline: +0.3%
Memory: โ 34.839MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.6%
โ medium_valid_headers_all
Time: โ 11.375ยตs (SLO: <20.000ยตs ๐ -43.1%) vs baseline: +0.5%
Memory: โ 34.918MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.0%
โ none_propagation_style
Time: โ 1.725ยตs (SLO: <10.000ยตs ๐ -82.8%) vs baseline: +0.5%
Memory: โ 34.996MB (SLO: <35.500MB ๐ก -1.4%) vs baseline: +5.3%
โ tracecontext_headers
Time: โ 34.559ยตs (SLO: <40.000ยตs ๐ -13.6%) vs baseline: -0.3%
Memory: โ 34.918MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.5%
โ valid_headers_all
Time: โ 6.594ยตs (SLO: <10.000ยตs ๐ -34.1%) vs baseline: +0.5%
Memory: โ 34.977MB (SLO: <35.500MB ๐ก -1.5%) vs baseline: +5.0%
โ valid_headers_basic
Time: โ 6.148ยตs (SLO: <10.000ยตs ๐ -38.5%) vs baseline: -0.3%
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.8%
โ wsgi_empty_headers
Time: โ 1.613ยตs (SLO: <10.000ยตs ๐ -83.9%) vs baseline: +0.5%
Memory: โ 34.977MB (SLO: <35.500MB ๐ก -1.5%) vs baseline: +5.0%
โ wsgi_invalid_priority_header
Time: โ 6.600ยตs (SLO: <10.000ยตs ๐ -34.0%) vs baseline: +0.4%
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.9%
โ wsgi_invalid_span_id_header
Time: โ 1.612ยตs (SLO: <10.000ยตs ๐ -83.9%) vs baseline: +0.9%
Memory: โ 34.918MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.1%
โ wsgi_invalid_tags_header
Time: โ 6.618ยตs (SLO: <10.000ยตs ๐ -33.8%) vs baseline: -0.4%
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.9%
โ wsgi_invalid_trace_id_header
Time: โ 6.695ยตs (SLO: <10.000ยตs ๐ -33.1%) vs baseline: +1.6%
Memory: โ 34.996MB (SLO: <35.500MB ๐ก -1.4%) vs baseline: +5.1%
โ wsgi_large_header_no_matches
Time: โ 28.902ยตs (SLO: <40.000ยตs ๐ -27.7%) vs baseline: -0.3%
Memory: โ 35.036MB (SLO: <35.500MB ๐ก -1.3%) vs baseline: +5.1%
โ wsgi_large_valid_headers_all
Time: โ 30.009ยตs (SLO: <40.000ยตs ๐ -25.0%) vs baseline: ~same
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.7%
โ wsgi_medium_header_no_matches
Time: โ 10.283ยตs (SLO: <20.000ยตs ๐ -48.6%) vs baseline: -0.1%
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.8%
โ wsgi_medium_valid_headers_all
Time: โ 11.762ยตs (SLO: <20.000ยตs ๐ -41.2%) vs baseline: ~same
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.1%
โ wsgi_valid_headers_all
Time: โ 6.633ยตs (SLO: <10.000ยตs ๐ -33.7%) vs baseline: +0.7%
Memory: โ 34.839MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.6%
โ wsgi_valid_headers_basic
Time: โ 6.169ยตs (SLO: <10.000ยตs ๐ -38.3%) vs baseline: +0.2%
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +5.0%
๐ก httppropagationinject - 16/16
โ ids_only
Time: โ 22.039ยตs (SLO: <30.000ยตs ๐ -26.5%) vs baseline: +4.8%
Memory: โ 34.996MB (SLO: <35.500MB ๐ก -1.4%) vs baseline: +5.0%
โ with_all
Time: โ 27.795ยตs (SLO: <40.000ยตs ๐ -30.5%) vs baseline: +0.2%
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.9%
โ with_dd_origin
Time: โ 24.806ยตs (SLO: <30.000ยตs ๐ -17.3%) vs baseline: +0.2%
Memory: โ 34.977MB (SLO: <35.500MB ๐ก -1.5%) vs baseline: +4.9%
โ with_priority_and_origin
Time: โ 24.220ยตs (SLO: <40.000ยตs ๐ -39.4%) vs baseline: ~same
Memory: โ 34.957MB (SLO: <35.500MB ๐ก -1.5%) vs baseline: +4.7%
โ with_sampling_priority
Time: โ 21.148ยตs (SLO: <30.000ยตs ๐ -29.5%) vs baseline: +0.3%
Memory: โ 34.819MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.5%
โ with_tags
Time: โ 26.164ยตs (SLO: <40.000ยตs ๐ -34.6%) vs baseline: +0.3%
Memory: โ 35.016MB (SLO: <35.500MB ๐ก -1.4%) vs baseline: +5.2%
โ with_tags_invalid
Time: โ 27.535ยตs (SLO: <40.000ยตs ๐ -31.2%) vs baseline: +0.1%
Memory: โ 34.937MB (SLO: <35.500MB ๐ก -1.6%) vs baseline: +4.6%
โ with_tags_max_size
Time: โ 26.573ยตs (SLO: <40.000ยตs ๐ -33.6%) vs baseline: -0.1%
Memory: โ 34.957MB (SLO: <35.500MB ๐ก -1.5%) vs baseline: +4.7%
๐ก otelspan - 22/22
โ add-event
Time: โ 40.220ms (SLO: <47.150ms ๐ -14.7%) vs baseline: +0.2%
Memory: โ 39.545MB (SLO: <47.000MB ๐ -15.9%) vs baseline: +5.0%
โ add-metrics
Time: โ 257.685ms (SLO: <344.800ms ๐ -25.3%) vs baseline: -1.6%
Memory: โ 43.895MB (SLO: <47.500MB -7.6%) vs baseline: +5.0%
โ add-tags
Time: โ 315.518ms (SLO: <321.000ms ๐ก -1.7%) vs baseline: -0.7%
Memory: โ 43.752MB (SLO: <47.500MB -7.9%) vs baseline: +4.9%
โ get-context
Time: โ 80.465ms (SLO: <92.350ms ๐ -12.9%) vs baseline: -0.1%
Memory: โ 39.979MB (SLO: <46.500MB ๐ -14.0%) vs baseline: +5.0%
โ is-recording
Time: โ 37.820ms (SLO: <44.500ms ๐ -15.0%) vs baseline: ~same
Memory: โ 39.523MB (SLO: <47.500MB ๐ -16.8%) vs baseline: +4.9%
โ record-exception
Time: โ 58.937ms (SLO: <67.650ms ๐ -12.9%) vs baseline: -0.3%
Memory: โ 40.000MB (SLO: <47.000MB ๐ -14.9%) vs baseline: +4.4%
โ set-status
Time: โ 44.233ms (SLO: <50.400ms ๐ -12.2%) vs baseline: -0.3%
Memory: โ 39.467MB (SLO: <47.000MB ๐ -16.0%) vs baseline: +4.7%
โ start
Time: โ 37.838ms (SLO: <43.450ms ๐ -12.9%) vs baseline: +1.7%
Memory: โ 39.492MB (SLO: <47.000MB ๐ -16.0%) vs baseline: +5.1%
โ start-finish
Time: โ 83.069ms (SLO: <88.000ms -5.6%) vs baseline: +0.3%
Memory: โ 37.375MB (SLO: <46.500MB ๐ -19.6%) vs baseline: +4.7%
โ start-finish-telemetry
Time: โ 84.450ms (SLO: <89.000ms -5.1%) vs baseline: -0.1%
Memory: โ 37.434MB (SLO: <46.500MB ๐ -19.5%) vs baseline: +4.9%
โ update-name
Time: โ 38.675ms (SLO: <45.150ms ๐ -14.3%) vs baseline: ~same
Memory: โ 39.583MB (SLO: <47.000MB ๐ -15.8%) vs baseline: +4.6%
๐ก ratelimiter - 12/12
โ defaults
Time: โ 2.364ยตs (SLO: <10.000ยตs ๐ -76.4%) vs baseline: +1.1%
Memory: โ 35.134MB (SLO: <35.500MB ๐ก -1.0%) vs baseline: +4.9%
โ high_rate_limit
Time: โ 2.399ยตs (SLO: <10.000ยตs ๐ -76.0%) vs baseline: -0.7%
Memory: โ 35.134MB (SLO: <35.500MB ๐ก -1.0%) vs baseline: +5.1%
โ long_window
Time: โ 2.372ยตs (SLO: <10.000ยตs ๐ -76.3%) vs baseline: +0.4%
Memory: โ 35.173MB (SLO: <35.500MB ๐ก -0.9%) vs baseline: +5.2%
โ low_rate_limit
Time: โ 2.360ยตs (SLO: <10.000ยตs ๐ -76.4%) vs baseline: +1.0%
Memory: โ 35.154MB (SLO: <35.500MB ๐ก -1.0%) vs baseline: +5.0%
โ no_rate_limit
Time: โ 0.832ยตs (SLO: <10.000ยตs ๐ -91.7%) vs baseline: -0.2%
Memory: โ 35.114MB (SLO: <35.500MB ๐ก -1.1%) vs baseline: +4.9%
โ short_window
Time: โ 2.494ยตs (SLO: <10.000ยตs ๐ -75.1%) vs baseline: ~same
Memory: โ 35.095MB (SLO: <35.500MB ๐ก -1.1%) vs baseline: +4.6%
๐ก recursivecomputation - 8/8
โ deep
Time: โ 308.654ms (SLO: <320.950ms -3.8%) vs baseline: -0.2%
Memory: โ 35.979MB (SLO: <36.500MB ๐ก -1.4%) vs baseline: +4.6%
โ deep-profiled
Time: โ 315.511ms (SLO: <359.150ms ๐ -12.2%) vs baseline: -0.2%
Memory: โ 39.892MB (SLO: <40.500MB ๐ก -1.5%) vs baseline: +4.9%
โ medium
Time: โ 6.993ms (SLO: <7.400ms -5.5%) vs baseline: -0.3%
Memory: โ 34.898MB (SLO: <35.500MB ๐ก -1.7%) vs baseline: +4.9%
โ shallow
Time: โ 0.945ms (SLO: <1.050ms ๐ -10.0%) vs baseline: +0.7%
Memory: โ 34.859MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +5.0%
๐ก samplingrules - 8/8
โ average_match
Time: โ 137.015ยตs (SLO: <290.000ยตs ๐ -52.8%) vs baseline: -0.6%
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +5.2%
โ high_match
Time: โ 174.183ยตs (SLO: <480.000ยตs ๐ -63.7%) vs baseline: +0.5%
Memory: โ 34.800MB (SLO: <35.500MB ๐ก -2.0%) vs baseline: +4.4%
โ low_match
Time: โ 99.356ยตs (SLO: <120.000ยตs ๐ -17.2%) vs baseline: +0.2%
Memory: โ 603.771MB (SLO: <700.000MB ๐ -13.7%) vs baseline: +4.9%
โ very_low_match
Time: โ 2.663ms (SLO: <8.500ms ๐ -68.7%) vs baseline: -0.3%
Memory: โ 71.139MB (SLO: <75.000MB -5.1%) vs baseline: +4.9%
๐ก sethttpmeta - 32/32
โ all-disabled
Time: โ 10.505ยตs (SLO: <20.000ยตs ๐ -47.5%) vs baseline: -1.0%
Memory: โ 35.271MB (SLO: <36.000MB -2.0%) vs baseline: +3.7%
โ all-enabled
Time: โ 41.210ยตs (SLO: <50.000ยตs ๐ -17.6%) vs baseline: +2.7%
Memory: โ 35.370MB (SLO: <36.000MB ๐ก -1.8%) vs baseline: +4.1%
โ collectipvariant_exists
Time: โ 40.978ยตs (SLO: <50.000ยตs ๐ -18.0%) vs baseline: +0.4%
Memory: โ 35.409MB (SLO: <36.000MB ๐ก -1.6%) vs baseline: +3.7%
โ no-collectipvariant
Time: โ 40.121ยตs (SLO: <50.000ยตs ๐ -19.8%) vs baseline: ~same
Memory: โ 35.350MB (SLO: <36.000MB ๐ก -1.8%) vs baseline: +4.0%
โ no-useragentvariant
Time: โ 38.828ยตs (SLO: <50.000ยตs ๐ -22.3%) vs baseline: -0.8%
Memory: โ 35.507MB (SLO: <36.000MB ๐ก -1.4%) vs baseline: +4.3%
โ obfuscation-no-query
Time: โ 40.799ยตs (SLO: <50.000ยตs ๐ -18.4%) vs baseline: +0.2%
Memory: โ 35.271MB (SLO: <36.000MB -2.0%) vs baseline: +3.7%
โ obfuscation-regular-case-explicit-query
Time: โ 76.056ยตs (SLO: <90.000ยตs ๐ -15.5%) vs baseline: +0.5%
Memory: โ 35.645MB (SLO: <36.500MB -2.3%) vs baseline: +4.7%
โ obfuscation-regular-case-implicit-query
Time: โ 76.677ยตs (SLO: <90.000ยตs ๐ -14.8%) vs baseline: ~same
Memory: โ 35.763MB (SLO: <36.500MB -2.0%) vs baseline: +4.9%
โ obfuscation-send-querystring-disabled
Time: โ 154.253ยตs (SLO: <170.000ยตs -9.3%) vs baseline: ~same
Memory: โ 35.665MB (SLO: <36.500MB -2.3%) vs baseline: +4.6%
โ obfuscation-worst-case-explicit-query
Time: โ 148.492ยตs (SLO: <160.000ยตs -7.2%) vs baseline: -0.3%
Memory: โ 35.743MB (SLO: <36.500MB -2.1%) vs baseline: +5.0%
โ obfuscation-worst-case-implicit-query
Time: โ 154.798ยตs (SLO: <170.000ยตs -8.9%) vs baseline: -0.2%
Memory: โ 35.763MB (SLO: <36.500MB -2.0%) vs baseline: +4.9%
โ useragentvariant_exists_1
Time: โ 39.668ยตs (SLO: <50.000ยตs ๐ -20.7%) vs baseline: +0.3%
Memory: โ 35.330MB (SLO: <36.000MB ๐ก -1.9%) vs baseline: +4.0%
โ useragentvariant_exists_2
Time: โ 40.645ยตs (SLO: <50.000ยตs ๐ -18.7%) vs baseline: -0.4%
Memory: โ 35.271MB (SLO: <36.000MB -2.0%) vs baseline: +3.7%
โ useragentvariant_exists_3
Time: โ 39.976ยตs (SLO: <50.000ยตs ๐ -20.0%) vs baseline: -0.3%
Memory: โ 35.291MB (SLO: <36.000MB ๐ก -2.0%) vs baseline: +3.7%
โ useragentvariant_not_exists_1
Time: โ 39.462ยตs (SLO: <50.000ยตs ๐ -21.1%) vs baseline: -0.7%
Memory: โ 35.311MB (SLO: <36.000MB ๐ก -1.9%) vs baseline: +3.7%
โ useragentvariant_not_exists_2
Time: โ 39.460ยตs (SLO: <50.000ยตs ๐ -21.1%) vs baseline: -0.2%
Memory: โ 35.468MB (SLO: <36.000MB ๐ก -1.5%) vs baseline: +4.4%
๐ก span - 26/26
โ add-event
Time: โ 18.166ms (SLO: <22.500ms ๐ -19.3%) vs baseline: +0.2%
Memory: โ 36.912MB (SLO: <53.000MB ๐ -30.4%) vs baseline: +4.8%
โ add-metrics
Time: โ 88.251ms (SLO: <93.500ms -5.6%) vs baseline: +0.4%
Memory: โ 41.130MB (SLO: <53.000MB ๐ -22.4%) vs baseline: +4.6%
โ add-tags
Time: โ 142.147ms (SLO: <155.000ms -8.3%) vs baseline: +0.2%
Memory: โ 41.204MB (SLO: <53.000MB ๐ -22.3%) vs baseline: +5.1%
โ get-context
Time: โ 16.861ms (SLO: <20.500ms ๐ -17.8%) vs baseline: ~same
Memory: โ 36.838MB (SLO: <53.000MB ๐ -30.5%) vs baseline: +4.8%
โ is-recording
Time: โ 17.280ms (SLO: <20.500ms ๐ -15.7%) vs baseline: +1.0%
Memory: โ 36.798MB (SLO: <53.000MB ๐ -30.6%) vs baseline: +4.8%
โ record-exception
Time: โ 36.336ms (SLO: <40.000ms -9.2%) vs baseline: +0.2%
Memory: โ 37.459MB (SLO: <53.000MB ๐ -29.3%) vs baseline: +5.3%
โ set-status
Time: โ 18.564ms (SLO: <22.000ms ๐ -15.6%) vs baseline: -0.5%
Memory: โ 36.740MB (SLO: <53.000MB ๐ -30.7%) vs baseline: +4.7%
โ start
Time: โ 17.231ms (SLO: <20.500ms ๐ -15.9%) vs baseline: +2.2%
Memory: โ 36.781MB (SLO: <53.000MB ๐ -30.6%) vs baseline: +4.9%
โ start-finish
Time: โ 50.981ms (SLO: <52.500ms -2.9%) vs baseline: -0.1%
Memory: โ 34.839MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.8%
โ start-finish-telemetry
Time: โ 52.257ms (SLO: <54.500ms -4.1%) vs baseline: ~same
Memory: โ 34.878MB (SLO: <35.500MB ๐ก -1.8%) vs baseline: +4.9%
โ start-finish-traceid128
Time: โ 53.892ms (SLO: <57.000ms -5.5%) vs baseline: -0.1%
Memory: โ 34.819MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.3%
โ start-traceid128
Time: โ 17.222ms (SLO: <22.500ms ๐ -23.5%) vs baseline: -0.2%
Memory: โ 36.781MB (SLO: <53.000MB ๐ -30.6%) vs baseline: +5.2%
โ update-name
Time: โ 17.317ms (SLO: <22.000ms ๐ -21.3%) vs baseline: +1.4%
Memory: โ 36.833MB (SLO: <53.000MB ๐ -30.5%) vs baseline: +4.7%
๐ก tracer - 6/6
โ large
Time: โ 29.166ms (SLO: <32.950ms ๐ -11.5%) vs baseline: +0.2%
Memory: โ 36.097MB (SLO: <36.500MB ๐ก -1.1%) vs baseline: +5.0%
โ medium
Time: โ 2.872ms (SLO: <3.200ms ๐ -10.3%) vs baseline: -0.3%
Memory: โ 34.839MB (SLO: <35.500MB ๐ก -1.9%) vs baseline: +4.9%
โ small
Time: โ 330.285ยตs (SLO: <370.000ยตs ๐ -10.7%) vs baseline: +1.6%
Memory: โ 34.800MB (SLO: <35.500MB ๐ก -2.0%) vs baseline: +4.8%
โ ๏ธ Unstable Tests (1 suite)
โ ๏ธ packagesupdateimporteddependencies - 24/24 (1 unstable)
โ import_many
Time: โ 155.363ยตs (SLO: <170.000ยตs -8.6%) vs baseline: ~same
Memory: โ 39.853MB (SLO: <43.000MB -7.3%) vs baseline: +5.0%
โ import_many_cached
Time: โ 121.274ยตs (SLO: <130.000ยตs -6.7%) vs baseline: +0.3%
Memory: โ 39.984MB (SLO: <43.000MB -7.0%) vs baseline: +5.0%
โ import_many_stdlib
Time: โ 0.755ms (SLO: <1.750ms ๐ -56.9%) vs baseline: -0.7%
Memory: โ 40.160MB (SLO: <43.000MB -6.6%) vs baseline: +5.6%
โ ๏ธ import_many_stdlib_cached
Time: โ ๏ธ 0.172ms (SLO: <1.100ms ๐ -84.3%) vs baseline: +0.6%
Memory: โ 39.907MB (SLO: <43.000MB -7.2%) vs baseline: +5.0%
โ import_many_unknown
Time: โ 829.347ยตs (SLO: <890.000ยตs -6.8%) vs baseline: -0.7%
Memory: โ 40.051MB (SLO: <43.000MB -6.9%) vs baseline: +4.9%
โ import_many_unknown_cached
Time: โ 790.578ยตs (SLO: <870.000ยตs -9.1%) vs baseline: ~same
Memory: โ 40.052MB (SLO: <43.000MB -6.9%) vs baseline: +5.0%
โ import_one
Time: โ 19.742ยตs (SLO: <30.000ยตs ๐ -34.2%) vs baseline: +0.2%
Memory: โ 39.824MB (SLO: <43.000MB -7.4%) vs baseline: +5.0%
โ import_one_cache
Time: โ 6.263ยตs (SLO: <10.000ยตs ๐ -37.4%) vs baseline: ~same
Memory: โ 39.986MB (SLO: <43.000MB -7.0%) vs baseline: +5.5%
โ import_one_stdlib
Time: โ 18.796ยตs (SLO: <20.000ยตs -6.0%) vs baseline: +0.1%
Memory: โ 39.904MB (SLO: <43.000MB -7.2%) vs baseline: +5.3%
โ import_one_stdlib_cache
Time: โ 6.307ยตs (SLO: <10.000ยตs ๐ -36.9%) vs baseline: +0.2%
Memory: โ 40.049MB (SLO: <43.000MB -6.9%) vs baseline: +5.4%
โ import_one_unknown
Time: โ 45.453ยตs (SLO: <50.000ยตs -9.1%) vs baseline: +0.2%
Memory: โ 40.113MB (SLO: <43.000MB -6.7%) vs baseline: +5.4%
โ import_one_unknown_cache
Time: โ 6.285ยตs (SLO: <10.000ยตs ๐ -37.2%) vs baseline: ~same
Memory: โ 40.059MB (SLO: <43.000MB -6.8%) vs baseline: +5.7%
โ All Tests Passing (5 suites)
โ iast_aspects - 40/40
โ re_expand_aspect
Time: โ 37.016ยตs (SLO: <40.000ยตs -7.5%) vs baseline: +6.1%
Memory: โ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.7%
โ re_expand_noaspect
Time: โ 34.719ยตs (SLO: <40.000ยตs ๐ -13.2%) vs baseline: -0.5%
Memory: โ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9%
โ re_findall_aspect
Time: โ 3.356ยตs (SLO: <10.000ยตs ๐ -66.4%) vs baseline: -1.0%
Memory: โ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +5.0%
โ re_findall_noaspect
Time: โ 3.259ยตs (SLO: <10.000ยตs ๐ -67.4%) vs baseline: -0.4%
Memory: โ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +4.9%
โ re_finditer_aspect
Time: โ 4.524ยตs (SLO: <10.000ยตs ๐ -54.8%) vs baseline: +0.4%
Memory: โ 41.465MB (SLO: <43.500MB -4.7%) vs baseline: +4.8%
โ re_finditer_noaspect
Time: โ 3.276ยตs (SLO: <10.000ยตs ๐ -67.2%) vs baseline: ~same
Memory: โ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.8%
โ re_fullmatch_aspect
Time: โ 2.756ยตs (SLO: <10.000ยตs ๐ -72.4%) vs baseline: ~same
Memory: โ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +4.9%
โ re_fullmatch_noaspect
Time: โ 3.018ยตs (SLO: <10.000ยตs ๐ -69.8%) vs baseline: -0.1%
Memory: โ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.7%
โ re_group_aspect
Time: โ 4.808ยตs (SLO: <10.000ยตs ๐ -51.9%) vs baseline: ~same
Memory: โ 41.465MB (SLO: <43.500MB -4.7%) vs baseline: +5.1%
โ re_group_noaspect
Time: โ 4.860ยตs (SLO: <10.000ยตs ๐ -51.4%) vs baseline: +0.7%
Memory: โ 41.504MB (SLO: <43.500MB -4.6%) vs baseline: +5.0%
โ re_groups_aspect
Time: โ 4.932ยตs (SLO: <10.000ยตs ๐ -50.7%) vs baseline: ~same
Memory: โ 41.465MB (SLO: <43.500MB -4.7%) vs baseline: +4.9%
โ re_groups_noaspect
Time: โ 4.942ยตs (SLO: <10.000ยตs ๐ -50.6%) vs baseline: +0.5%
Memory: โ 41.484MB (SLO: <43.500MB -4.6%) vs baseline: +4.9%
โ re_match_aspect
Time: โ 2.801ยตs (SLO: <10.000ยตs ๐ -72.0%) vs baseline: -0.7%
Memory: โ 41.425MB (SLO: <43.500MB -4.8%) vs baseline: +4.9%
โ re_match_noaspect
Time: โ 3.098ยตs (SLO: <10.000ยตs ๐ -69.0%) vs baseline: +0.5%
Memory: โ 41.465MB (SLO: <43.500MB -4.7%) vs baseline: +5.2%
โ re_search_aspect
Time: โ 2.635ยตs (SLO: <10.000ยตs ๐ -73.6%) vs baseline: +0.3%
Memory: โ 41.347MB (SLO: <43.500MB -5.0%) vs baseline: +4.6%
โ re_search_noaspect
Time: โ 2.882ยตs (SLO: <10.000ยตs ๐ -71.2%) vs baseline: +0.2%
Memory: โ 41.425MB (SLO: <43.500MB -4.8%) vs baseline: +4.9%
โ re_sub_aspect
Time: โ 3.475ยตs (SLO: <10.000ยตs ๐ -65.2%) vs baseline: -0.5%
Memory: โ 41.347MB (SLO: <43.500MB -5.0%) vs baseline: +4.8%
โ re_sub_noaspect
Time: โ 4.009ยตs (SLO: <10.000ยตs ๐ -59.9%) vs baseline: +0.9%
Memory: โ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9%
โ re_subn_aspect
Time: โ 3.735ยตs (SLO: <10.000ยตs ๐ -62.6%) vs baseline: +0.8%
Memory: โ 41.327MB (SLO: <43.500MB -5.0%) vs baseline: +4.7%
โ re_subn_noaspect
Time: โ 4.079ยตs (SLO: <10.000ยตs ๐ -59.2%) vs baseline: -0.1%
Memory: โ 41.366MB (SLO: <43.500MB -4.9%) vs baseline: +4.5%
โ iastaspectssplit - 12/12
โ rsplit_aspect
Time: โ 1.700ยตs (SLO: <10.000ยตs ๐ -83.0%) vs baseline: +9.7%
Memory: โ 41.504MB (SLO: <43.500MB -4.6%) vs baseline: +5.3%
โ rsplit_noaspect
Time: โ 1.623ยตs (SLO: <10.000ยตs ๐ -83.8%) vs baseline: ~same
Memory: โ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.7%
โ split_aspect
Time: โ 1.564ยตs (SLO: <10.000ยตs ๐ -84.4%) vs baseline: -0.4%
Memory: โ 41.465MB (SLO: <43.500MB -4.7%) vs baseline: +5.1%
โ split_noaspect
Time: โ 1.645ยตs (SLO: <10.000ยตs ๐ -83.6%) vs baseline: +1.6%
Memory: โ 41.406MB (SLO: <43.500MB -4.8%) vs baseline: +4.9%
โ splitlines_aspect
Time: โ 1.525ยตs (SLO: <10.000ยตs ๐ -84.8%) vs baseline: -0.6%
Memory: โ 41.445MB (SLO: <43.500MB -4.7%) vs baseline: +4.9%
โ splitlines_noaspect
Time: โ 1.559ยตs (SLO: <10.000ยตs ๐ -84.4%) vs baseline: +0.5%
Memory: โ 41.386MB (SLO: <43.500MB -4.9%) vs baseline: +4.7%
โ iastpropagation - 8/8
โ no-propagation
Time: โ 48.838ยตs (SLO: <60.000ยตs ๐ -18.6%) vs baseline: ~same
Memory: โ 38.240MB (SLO: <42.000MB -9.0%) vs baseline: +5.0%
โ propagation_enabled
Time: โ 136.085ยตs (SLO: <190.000ยตs ๐ -28.4%) vs baseline: +1.0%
Memory: โ 38.280MB (SLO: <42.000MB -8.9%) vs baseline: +4.9%
โ propagation_enabled_100
Time: โ 1.579ms (SLO: <2.300ms ๐ -31.4%) vs baseline: +0.3%
Memory: โ 38.280MB (SLO: <42.000MB -8.9%) vs baseline: +5.0%
โ propagation_enabled_1000
Time: โ 29.097ms (SLO: <34.550ms ๐ -15.8%) vs baseline: +0.3%
Memory: โ 38.260MB (SLO: <42.000MB -8.9%) vs baseline: +4.9%
โ otelsdkspan - 24/24
โ add-event
Time: โ 40.592ms (SLO: <42.000ms -3.4%) vs baseline: +0.7%
Memory: โ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.7%
โ add-link
Time: โ 36.310ms (SLO: <38.550ms -5.8%) vs baseline: -0.2%
Memory: โ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.6%
โ add-metrics
Time: โ 220.902ms (SLO: <232.000ms -4.8%) vs baseline: +0.4%
Memory: โ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.7%
โ add-tags
Time: โ 213.405ms (SLO: <221.600ms -3.7%) vs baseline: ~same
Memory: โ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +5.2%
โ get-context
Time: โ 28.974ms (SLO: <31.300ms -7.4%) vs baseline: -0.5%
Memory: โ 37.749MB (SLO: <39.000MB -3.2%) vs baseline: +5.2%
โ is-recording
Time: โ 29.362ms (SLO: <31.000ms -5.3%) vs baseline: +0.7%
Memory: โ 37.611MB (SLO: <39.000MB -3.6%) vs baseline: +4.9%
โ record-exception
Time: โ 63.186ms (SLO: <65.850ms -4.0%) vs baseline: +0.2%
Memory: โ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +5.0%
โ set-status
Time: โ 31.870ms (SLO: <34.150ms -6.7%) vs baseline: ~same
Memory: โ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +5.1%
โ start
Time: โ 29.155ms (SLO: <30.150ms -3.3%) vs baseline: +0.9%
Memory: โ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.8%
โ start-finish
Time: โ 34.149ms (SLO: <35.350ms -3.4%) vs baseline: -0.1%
Memory: โ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +4.7%
โ start-finish-telemetry
Time: โ 33.843ms (SLO: <35.450ms -4.5%) vs baseline: -1.1%
Memory: โ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.6%
โ update-name
Time: โ 31.139ms (SLO: <33.400ms -6.8%) vs baseline: +0.4%
Memory: โ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +5.0%
โ packagespackageforrootmodulemapping - 4/4
โ cache_off
Time: โ 341.539ms (SLO: <354.300ms -3.6%) vs baseline: -1.2%
Memory: โ 40.757MB (SLO: <43.500MB -6.3%) vs baseline: +3.6%
โ cache_on
Time: โ 0.385ยตs (SLO: <10.000ยตs ๐ -96.1%) vs baseline: +0.8%
Memory: โ 39.501MB (SLO: <43.000MB -8.1%) vs baseline: +3.8%
โน๏ธ Scenarios Missing SLO Configuration (26 scenarios)
The following scenarios exist in candidate data but have no SLO thresholds configured:
coreapiscenario-core_dispatch_listenerscoreapiscenario-core_dispatch_no_listenerscoreapiscenario-core_dispatch_with_results_listenerscoreapiscenario-core_dispatch_with_results_no_listenersdjangosimple-baselineerrortrackingdjangosimple-baselineerrortrackingflasksqli-baselineflasksimple-baselineflasksqli-baselinesethttpmeta-obfuscation-disabledstartup-baselinestartup-baseline_djangostartup-baseline_flaskstartup-ddtrace_runstartup-ddtrace_run_appsecstartup-ddtrace_run_profilingstartup-ddtrace_run_runtime_metricsstartup-ddtrace_run_send_spanstartup-ddtrace_run_telemetry_disabledstartup-ddtrace_run_telemetry_enabledstartup-import_ddtracestartup-import_ddtrace_autostartup-import_ddtrace_auto_djangostartup-import_ddtrace_auto_flaskstartup-import_ddtrace_djangostartup-import_ddtrace_flask
@codex review
To use Codex here, create a Codex account and connect to github.
@codex review
To use Codex here, create a Codex account and connect to github.
Looks good to me. Aren't we adding the "annotated" check too when setting the prompt through annotate.
@yahya-mouman Added the annotated tag :+1: