experiment: add span links option

[wconti27]

Motivation

Span links are generally available. Given that span links do not modify the trace structure (unlike parent-child propagation), we should be able to add links to any messaging consumer spans by default if a context is found in the message without this being a breaking change. We should also always inject tracing context (possibly with an explicit option to turn off injection to save bytes through queues), since our ideal state is to always connect distributed contexts.

Pros of this PR:

• Distributed context is always maintained on a consumer trace (if available), allowing users to follow a consume trace upstream
• No-Touch solution: No need for setting DD_[INTEGRATION]_PROPAGATION_ENABLED (many integrations where propagation was added later rely on this config as enabling by default is a breaking change, such as Kafka and AWS). Also, a quick github search shows 0 github repositories outside of Datadog using this setting for Kafka python.
• Wider use-cases than parent-child propagation: Span links allow us to support distributed contexts for Fan-in and Fan-Out scenarios which previously were not supportable
• On the consumer side of things, I see no risks in always using Span Links by default. Even if the consumer is also using parent-child propagation, there is no harm in also setting a link to the producer span.

Cons of this PR:

• Span Links are only useful if trace context is injected to the carrier. Many integrations are configured to not currently inject context by default, is it a breaking change to change them to always inject?
• More bytes through messaging queues if we always inject, users may complain

Checklist

• [ ] PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• [ ] Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions[bot]]

CODEOWNERS have been resolved as:

ddtrace/_trace/trace_handlers.py                                        @DataDog/apm-sdk-api-python
ddtrace/_trace/utils.py                                                 @DataDog/apm-sdk-api-python
ddtrace/contrib/internal/botocore/patch.py                              @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/botocore/services/kinesis.py                   @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/botocore/services/sqs.py                       @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/kafka/patch.py                                 @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/trace_utils.py                                          @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/settings/config.py                                              @DataDog/python-guild @DataDog/apm-sdk-api-python

[pr-commenter[bot]]

Benchmarks

Benchmark execution time: 2024-10-18 22:29:01

Comparing candidate commit 0e8835cbce770195856854a3f0e0cc9ae59275f3 in PR branch conti/implement-span-links-for-autoinstrumentation with baseline commit a201fc0680ffd3005e60345fc1c3cd8eb960e58c in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 365 metrics, 53 unstable metrics.

[datadog-dd-trace-py-rkomorn[bot]]

Datadog Report

Branch report: conti/implement-span-links-for-autoinstrumentation Commit report: 0e8835c Test service: dd-trace-py

:white_check_mark: 0 Failed, 1129 Passed, 157 Skipped, 25m 22.84s Total duration (10m 59.01s time saved)

[github-actions[bot]]

This pull request has been automatically closed after a period of inactivity. After this much time, it will likely be easier to open a new pull request with the same changes than to update this one from the base branch. Please comment or reopen if you think this pull request was closed in error.