opentelemetry-collector-contrib

opentelemetry-collector-contrib copied to clipboard

The purpose and use-cases of the new component

We need a receiver to collect the Kubernetes objects and send them as log data(similar to fluent-plugin-kubernetes-objects fluentd plugin).

• It should collect objects in 2 ways
  • Pull: Collects all available objects at an interval by calling the list APIs.
  • Watch: Collects only the new objects when they appear by calling the watch APIs.
    • More about watch API: https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes
• It should allow filtering objects using namespace, label selector and field selector

Example configuration for the component

receivers:
  k8sobjects: 
    objects:
        - name: <resource-name>
          mode: pull/watch
          interval: <interval> # Required if pull mode
          label_selector: <label selector>
          field_selector: <field selector>
          namespaces: [<namespace>,...] # all namespace if empty
        ...

receivers:
  k8sobjects:
    objects:
      - name: pods
        mode: pull
        namespace: [default]
        interval: 30s
      - name: event
        mode: watch
      - name: deployments
        mode: pull
        interval: 30s
        label_selector: app=myapp

Telemetry data types supported

logs

Is this a vendor-specific component?

• [ ] This is a vendor-specific component
• [ ] If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

No response

Additional context

Design doc: https://docs.google.com/document/d/13ZESiUe0IP73sMNHQT5ZmWo9BPPP_1sy385QVIgNVkw

@dmitryax should I add you as a sponsor?

This looks interesting. While I can't commit as a sponsor, I can volunteer as a "second-in-line" approver.

I can sponsor the component

@harshit-splunk I would recommend adopting the new Events API that was introduced in https://github.com/open-telemetry/opentelemetry-specification/pull/2676

I believe it doesn't conflict with your design doc, we just need to add event.domain and event.name as defined in https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/semantic_conventions/events.md

@harshit-splunk @dmitryax is this supposed to include pod logs? What does pull of pods and deployment in your example configuration refer to?

@scheler It doesn't collect pod logs. It collects data from Kubernetes API Server. It will be the same as kubectl get pod <some-pod> -oyaml

Ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/