Improve Granularity for SQL Databases

[AlexanderWert]

There are two parts to improving the granularity for SQL Databases:

• adding new fields to agent-server protocol, for now it's mostly to future-proof the captured data as we have to keep existing fields for compatibility with UI.
• capture values for those new fields on SQL Databases, which is actually providing better granularity to the end-user.

Each step can be completed independently, but this task will be considered complete when both parts are completed.

First step : add new service fields

• Add and capture values for span.context.service.target.type and span.context.service.target.name for exit spans.
• Infer from those new fields the value of span.destination.service.resource and keep sending it.
• Add service_target_* fields to dropped spans metrics: add service_target_type and service_target_name next to destination_service_resource in transaction.dropped_spans_stats array, the related metrics documents should include span.service.target.type and span.service.target.name fields.
• Handle span compression with new fields (stop relying on resource internally)

Second step : capture values for new fields

---

Spec issue

• [x] https://github.com/elastic/apm/issues/622

Agent issues

• [x] https://github.com/elastic/apm-agent-java/issues/2541
• [x] https://github.com/elastic/apm-agent-dotnet/issues/1664
• [x] https://github.com/elastic/apm-agent-go/issues/1233
• [x] https://github.com/elastic/apm-agent-nodejs/issues/2621
• [x] https://github.com/elastic/apm-agent-python/issues/1497
• [ ] https://github.com/elastic/apm-agent-ruby/issues/1244
• [x] https://github.com/elastic/apm-agent-php/issues/647

[estolfo]

@AlexanderWert What if the agent doesn't detect the apm server version, how can it know which field to send?

[felixbarny]

I think agents should default to sending all three fields and if they detect that the APM Server they're connected to is new, they MAY drop the old field. But it's also totally fine to always send all fields, IMO.

[estolfo]

Thanks, @felixbarny. @AlexanderWert if you agree, I'll update the description to reflect Felix's input.

[felixbarny]

Ideally, I think the description and this discussion should happen on the spec PR. Otherwise, it may get out-of-sync.