[Meta] Support Kubernetes Obervability

[gizas]

This is the Meta issue related to the Proposal of https://docs.google.com/document/d/1dTfSYfcWl6aoFTOFQ2901XC-4yG1ai7cywd-NxJ5GdY/edit?pli=1

Context

Observability represents a more advanced stage of monitoring and this issue will focus on the Kubernetes Observability case, trying to provide logical abstraction to our users to be able to make general questions and our product to be able to reply with meaningful information and starting points of deeper analysis. This implies an underlying analysis and means to perform such complex queries to different blocks of information related to Kubernetes Resources

Proof of Concept:

In order to approach the solution for this proposal, we plan to start with a Proof of Concept (POC) that will demonsrtate the steps needed for:

• Defining the kind of resource to monitor. A small subset for monitored K8s resources is needed for this POC
• Define subset of questions that will be covered for kind of K8s resources
• Provide the queries that will be needed in order to answer above subset of questions
• Provide single endpoint that will be able to group all needed queries to answer specific questions. This endpoint aims to be an enhancement of current APIs available in Kibana and Elasticsearch
• Investigate the way for a user to call single endpoint for answering questions with general parameters and with ability to perform general logical questions

Stories List

A more detailed list of stories defined until now is:

### Tasks
- [ ] **Story 1**: Document full list of Kubernetes resources and relevant questions that concern each kind. We need questions per SLI category. Identify the best format for grouping information and questions
- [ ] **Story 2**: Define API schema for kubernetes observability. This is only the definition story and includes all the related work needed to be done to define type of queries, basic parameters and possible grouping that we can do. The definition should follow -[OpenAPI specification](https://swagger.io/docs/specification/about/)
- [ ] **Story 3**: List the queries that need to be done per question defined. Provide Logs, Metrics and Traces for each general question defined above.
- [ ] **Story 4**: Implement Kubernetes API according to schema defined before. API should be able to answer the basic questions related to Kubernetes Assets that also have been provided in previous tories. Important Note: Here we need to investigate current available solutions to perform queries in Kibana or Elasticsearch and enhance those if they miss basic functionality
- [ ] **Story 5**: Provide the enhancement for user questions that finally will leverage the underlying API. Investigate Kibana Plugins, AI Assistant or ay other possible solution we can find

[gizas]

For the Story List provided above:

Story 1: https://docs.google.com/document/d/1WGFNKg2OhjVt4w-ek67Epj4BtZBZ52rdlprcbIPR1Ok/edit

Story 2: https://github.com/elastic/beats/issues/39717

Story 3: https://docs.google.com/document/d/1nucHb2kOUNFDAD3pnqb24iQvzT_KTrhupHZ45q6Vz64/edit#heading=h.ol1sqvhc01na

Story 4: https://github.com/elastic/kibana/pull/183036

[gizas]

Moving to another repository