datadog-agent icon indicating copy to clipboard operation
datadog-agent copied to clipboard

[BUG] failed to send, dropping xxx traces to intake at xxx after 3 retries ([Errno 111] Connection refused)

Open sachams opened this issue 2 years ago • 4 comments

Agent Environment

  • Agent version 3.30.8
  • Python ddtrace version 1.12.6
	"sourceLocation": {
		"file": "/usr/local/lib/python3.10/site-packages/ddtrace/internal/writer.py",
		"line": 464,
		"function": "flush_queue"
	}

Describe what happened:

We frequently see this error in all our environments, for example:

failed to send, dropping 23 traces to intake at http://10.154.0.10:8126/v0.5/traces after 3 retries ([Errno 111] Connection refused)

DataDog is otherwise functioning fine. Would like to stop these errors being generated though.

Describe what you expected:

No errors.

Steps to reproduce the issue:

  • Deploy datadog helm 3.30.8
  • Use ddtrace in a Python application

Additional environment details (Operating System, Cloud provider, etc):

  • Running on python:3.10.10-slim-bullseye on GKE

sachams avatar May 25 '23 16:05 sachams

U should see happening datadog agent logs at this time. Highly likely your agent is rapidly restarting, because each time it exits for some reason

darkwind-noveo avatar Jun 08 '23 11:06 darkwind-noveo

I'm having the same, would like to stop it too

ricardodani avatar Jul 21 '23 20:07 ricardodani

Same issue here, @sachams have you found a fix for this?

vkaracic avatar Feb 27 '24 11:02 vkaracic

bump

luciidlou avatar Jul 03 '24 03:07 luciidlou

Do we have any ETA for this to be solved or a workaround?

miguelgf avatar Jul 09 '24 07:07 miguelgf

+1

jmborsani avatar Jul 22 '24 20:07 jmborsani

please fix

domino14 avatar Jul 29 '24 18:07 domino14