chore(tracing): add kafka cluster id to spans and DSM metrics

[wconti27]

trafficstars

Adds Kafka cluster ID as a span tag and within DSM metrics.

Checklist

• [x] PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• [x] Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions[bot]]

CODEOWNERS have been resolved as:

tests/contrib/kafka/meta.properties                                     @DataDog/apm-core-python @DataDog/apm-idm-python
.gitlab/services.yml                                                    @DataDog/python-guild @DataDog/apm-core-python
ddtrace/contrib/internal/kafka/patch.py                                 @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/ext/kafka.py                                                    @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/internal/datastreams/kafka.py                                   @DataDog/apm-core-python
docker-compose.yml                                                      @DataDog/apm-core-python
tests/contrib/kafka/test_kafka.py                                       @DataDog/apm-core-python @DataDog/apm-idm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_analytics_with_rate.json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_analytics_without_rate.json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_commit.json         @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_commit_with_consume_single_message.json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_commit_with_consume_with_error[False].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_commit_with_consume_with_multiple_messages.json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_commit_with_offset.json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_commit_with_only_async_arg.json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_message[False].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_message[True].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_schematized_span_service_and_operation[None-None].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_schematized_span_service_and_operation[None-mysvc].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_schematized_span_service_and_operation[v0-None].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_schematized_span_service_and_operation[v0-mysvc].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_schematized_span_service_and_operation[v1-None].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_schematized_span_service_and_operation[v1-mysvc].json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_service_override.json  @DataDog/apm-python
tests/snapshots/tests.contrib.kafka.test_kafka.test_service_override_env_var.json  @DataDog/apm-python

[datadog-dd-trace-py-rkomorn[bot]]

Datadog Report

Branch report: conti/kafka-cluster-id Commit report: 71074a8 Test service: dd-trace-py

:white_check_mark: 0 Failed, 592 Passed, 694 Skipped, 18m 57.58s Total duration (17m 59.94s time saved)

[pr-commenter[bot]]

Benchmarks

Benchmark execution time: 2024-10-24 16:51:29

Comparing candidate commit 71074a8c56c52d5165c17d01a560b003c7700cd7 in PR branch conti/kafka-cluster-id with baseline commit f26e7c4ce6e029147b657ef4daf2b758ac9d3410 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 388 metrics, 2 unstable metrics.