[BUG]: LLMObs agentLessEnabled does not mean no agent?

[tjk]

Tracer Version(s)

5.35.0

Node.js Version(s)

22.13.1

Bug Report

Cannot send LLMObs events using our agent configuration.

https://github.com/DataDog/dd-trace-js/blob/master/packages/dd-trace/src/llmobs/writers/spans/agentProxy.js looks like it expects an HTTP proxy?

Maybe the naming is confusing? agentLessEnabled just means without agent PROXY? We run with the agent...

Reproduction Code

We run our process with:

DD_DOGSTATSD_SOCKET=...
DD_APM_RECEIVER_SOCKET=...

And then:

tracer.init({
  llmobs: {
    mlApp: "...",
  },
})

And get error below, what are we doing wrong? Is UDS agent not supported?

Error Logs

Error sending 11 LLMObs span events to unix:127.0.0.1:8126evp_proxy/v2/api/v2/llmobs: connect ENOENT 127.0.0.1:8126evp_proxy/v2/api/v2/llmobs

Tracer Config

No response

Operating System

Amazon Linux

Bundling

ESBuild

[sabrenner]

Hi @tjk, thanks for opening this issue! The agentlessEnabled configuration for LLM Observability means LLMObs will send data in an agentless fashion - other data from the tracer will still use the agent if it's available.

I do think what you're running into is unexpected, which I will put in a fix for.

However, in the meantime, are you able to set agentlessEnabled to true via

tracer.init({
  llmobs: {
    mlApp: "...",
    agentlessEnabled: true
  },
})

and see if it resolves your issue? It should in this case send APM data through your agent and LLMObs data in an agentless fashion.

[tjk]

I did set it to agentLessEnabled: true and am getting data sent to DataDog (however I only see it in LLM dashboards and not in the LLM Monitoring section (traces, etc.) -- which could be a separate issue as well?).

[sabrenner]

Thanks for confirming - do you still see the

Error sending 11 LLMObs span events to...

message, or is there a different message in its place?

[tjk]

No. I get the successfully sent span events messages in its place.

[sabrenner]

Are there any other error/warning logs in general that get outputted? Additionally, is this only happening in your deployment environment, or are you able to repro this locally? If so, a small repro script would be helpful!

Feel free to open a support ticket here and mention me by name w/ this GH issue linked if providing more details here isn't possible. It's likely that since data is making it out of the SDK client, but not showing up in DD LLM Observability, that we might need some DD account details to figure this one out. Thanks!

[sabrenner]

Hi @tjk - apologies for the radio silence on this one! We recently released a change that might address this issue in version 5.52.0. Let me know if it helps at all (turning agentlessEnabled: false or even omitting it entirely). Thanks!

[sabrenner]

closing out this issue as i think the mentioned release should have resolved the incompatibility with LLM Observability not being able to send spans via UDS. please reopen and follow up otherwise!