Cannot decode v0.4 traces payload: msgp: too few bytes left to read object

[rkdrnf]

Describe what happened:

Cannot decode v0.4 traces payload: msgp: too few bytes left to read object

Sometimes above error occurs on datadog agent container. Usually it just works fine without any error.

ERROR:ddtrace.internal.writer:failed to send traces to Datadog Agent at http://datadog-agent:8126: HTTP error status 400, reason Bad Request

ddtrace client error occurs as a result.

Describe what you expected: There should be no error on both datadog agent container and ddtrace client.

Additional environment details (Operating System, Cloud provider, etc): Running datadog agent docker image gcr.io/datadoghq/agent:7 AWS ElasticBeanstalk Linux AMI 2 Docker Platform

Client: Python 3.6 Flask Running ddtrace with following uwsgi options

enable-threads=true
lazy-apps=true
import=ddtrace.bootstrap.sitecustomize

[florianmutter]

Did someone find out where this comes from? We also see a lot of the Cannot decode logs but I have no idea which component sends the invalid traces.

[s7an-it]

I get that error on tracerAgent.

[rahul-paigavan]

Observing similar issue - is there any impact of this on dropped traces/spans??

[peterdeme]

Anyone figured this one out?