Inconsistencies in health metrics

[deadok22]

I believe there are several inconsistencies in the health metrics reported here:

1. datadog.tracer.http.requests is emitted both as a count and as a distribution.
2. datadog.tracer.http.requests, the count metric, is incremented once per flush, while its distribution counterpart is incremented once per attempt - see how _send_payload is called recursively
3. datadog.tracer.http.sent.bytes is emitted as a distribution in two places: here and here. The semantics of the two invocations differ: the former emits a distribution metric once per payload, the latter - once per successful payload submission. The .sum component of the distribution is 2x the payload size on successful submissions, and is 1x the payload size on unsuccessful ones.

These metrics need some TLC

Which version of dd-trace-py are you using?

The issues are present on the master branch (commit efe93da7ee33f12d29aef3c717dde8fd70091698)

[Kyle-Verhoog]

Hey @deadok22 thanks for finding and detailing! All 3 points look like bugs to me. Looks like some stuff wasn't ported properly when the writer was refactored 😞.

Going to try to get a fix in for this by the end of the week!

[Kyle-Verhoog]

the health metrics feature was never fully GA'd and will be superseded by the telemetry effort