[receiver/k8sobjectreceiver]: Auto discovery of k8s resources(Pods, deployments, replicasets, etc)

[PradeepThota95]

Component(s)

receiver/k8sobjects

Is your feature request related to a problem? Please describe.

I am unsure about it; I do not know if this is the place to ask for this requirement. Please feel free to close it if it does not fall under the purview of OTel by suggesting alternatives Can I use OpenTelemetry receivers, such as K8s object receivers, to discover K8s resources such as workloads(Pods, deployments, etc.), services(all networking things), storage(all storage things), etc.? Is there any way I can pass this discovery info on to the exporter and then the backend, which helps build UI with all discovered resources?

Describe the solution you'd like

I do not have a solution, and I was wondering how to solve resource discovery using pure OTel.

Describe alternatives you've considered

External tools such as kubeview and kubewatch provide UI inbuilt into them.

Additional context

I was wondering if OTel would support or add a feature for K8s resource discovery mentioned above.

[github-actions[bot]]

Pinging code owners:

• receiver/k8sobjects: @dmitryax @hvaghani221 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[developer1622]

@PradeepThota95 Thanks for bringing it up. Even I, too, have the same concern.

In addition, how can we use the K8s Object Receiver to derive service maps and complete end-to-end relationships among all K8s resources(workloads, storage, networking,etc)?

Thank you.

[ChrsMark]

Hey @PradeepThota95 @developer1622 . I would suggest start trying some of the examples that the k8sobjects receiver provides in the docs: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/k8sobjectsreceiver.

Then you can configure the Collector to send to a backend or even write the output to a file and see how you could consume this information for your specific use case.

[PradeepThota95]

Thanks for the response @ChrsMark .

As per the docs, I've tried multiple examples and found that it is giving update as a Otel log related to the k8s resources (such as Pods, Deployments etc.)

For example, the formatted and edited json can found here Pod Response Log

However is there any way to get the service map relationship among k8s objects in the cluster using this receiver such as , a deployment has this many replicas, this many pods, storage relationship, etc.

Is this receiver is somehow extendable where I can write some handler kind of thing?

Thank you

[ChrsMark]

With a very basic config like the following:

k8sobjects:
  auth_type: serviceAccount
  objects:
    - name: pods
      mode: pull
      field_selector: status.phase=Running
      interval: 15m

and logging the data into the console through the debug exporter I'm able to get Pod objects like the following:

{
  "apiVersion": "v1",
  "kind": "Pod",
  "metadata": {
    "annotations": {
      "components.gke.io/component-name": "pdcsi",
      "components.gke.io/component-version": "0.18.7"
    },
    "creationTimestamp": "2024-05-20T08:16:45Z",
    "generateName": "pdcsi-node-",
    "labels": {
      "controller-revision-hash": "679648b9f7",
      "k8s-app": "gcp-compute-persistent-disk-csi-driver",
      "pod-template-generation": "1"
    },
    "managedFields": [],
    "name": "pdcsi-node-h2btb",
    "namespace": "kube-system",
    "ownerReferences": [
      {
        "apiVersion": "apps/v1",
        "blockOwnerDeletion": true,
        "controller": true,
        "kind": "DaemonSet",
        "name": "pdcsi-node",
        "uid": "935491b5-498f-454c-876a-4d7e9968a0e9"
      }
    ],
    "resourceVersion": "1381",
    "uid": "e970499c-b5f8-4a3f-9e33-e59fa296d8e2"
  },
  "spec": {},
  "status": {}
}

(I remove the spec and status for simplicity)

In this example you could leverage the ownerReferences values and built a relationship map on your own I guess. You can build any kind of relationship map you want based on the information that the raw objects can provide. As far as I know the receiver does not do any kind of sophisticated correlations which that wouldn't really be within the scope of a receiver.

[PradeepThota95]

Thanks you very much for the response @ChrsMark . So are you suggesting to write the custom exporter?

[github-actions[bot]]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/k8sobjects: @dmitryax @hvaghani221 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions[bot]]

This issue has been closed as inactive because it has been stale for 120 days with no activity.