[OTel] Support Serializing nested data

[pickypg]

This adds support for map[string]any and also arrays in general (as opposed to just slices) to the converted map.

Like mapstr.M and []mapstr.M, this will dive into the map[string]any and []map[string]any objects to ensure that they are properly converted for the pdata code.

Proposed commit message

Properly handles nested map[string]any and array types (not just slices) for OTel conversion to avoid Invalid type warnings from OTel.

Checklist

• [x] My code follows the style guidelines of this project
• [x] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

This should make the code more flexible -- not less.

Author's Checklist

• [ ] Ensure nested types are serialized.

How to test this PR locally

The easiest way is to inspect the new test and try removing the updated code to see what changes.

Related issues

• Closes https://github.com/elastic/beats/issues/45007

Use cases

Metricsets can now ship nested map[string]any that contain arrays and slices, including of []map[string]any.

[github-actions[bot]]

:robot: GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify[bot]]

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @pickypg? 🙏. For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[mauri870]

Added Vihas as a reviewer because of his recent work on otelmap.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[pickypg]

Please wait for at least another review.

No problem waiting. I am marking as auto-merge since after your review it's not going to do it on its own so that it can end up in the nightly build on the next pass.

[github-actions[bot]]

@mergifyio backport 8.17 8.18 8.19 9.0

[mergify[bot]]

backport 8.17 8.18 8.19 9.0

✅ Backports have been created

• #45024 [8.17](backport #45008) [OTel] Support Serializing nested data has been created for branch 8.17 but encountered conflicts
• #45025 [8.18](backport #45008) [OTel] Support Serializing nested data has been created for branch 8.18 but encountered conflicts
• #45026 [8.19](backport #45008) [OTel] Support Serializing nested data has been created for branch 8.19
• #45027 [9.0](backport #45008) [OTel] Support Serializing nested data has been created for branch 9.0