[Infra Monitoring UI] Support Containers from multiple module sources

[miltonhultgren]

As far as we are aware, we have 3 modules that collect metirc data for containers, namely the docker, kubernetes and containerd modules. On the UI side, our Inventory and Node Details page for Containers only supports reading data from docker module documents while the newer ContainerMetricsTable component only support reading from the kubernetes module documents.

Open questions

• Do these 3 modules overlap in which fields they populate, are there ECS fields to align on for shared metrics?
• Can we find equivalent metrics in different fields (something like Kubernetes limits might be hard) or do we accept that there is some overlap (ECS) and some that will be module specific? How do we represent that in the UIs?
• Do we want to create individual pages/tables for each module or instead configure which "mode" to use by props (let the page/table keep a set of fields to read per module, and perhaps allow these to be fairly different from each other)?
• Are there any other pages in the UI that reads Container data, and from which sources?
• What are the integrations teams view on this?

AC

There exists a plan for how to support these multiple sources.

[elasticmachine]

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

[miltonhultgren]

Similar to the issue in the Node Details page, we might also want to support containers instrumented with the containerd module .

[miltonhultgren]

I don't think this is something we're ready to work on, pulling it from the current cycle and will reshape to address the larger picture.

[roshan-elastic]

Hey everyone,

Looking at this to get an understanding of the issue here. Replaying back the problem to check I understand correctly:

Current State

APM > Infrastructure (beta) tab

If a user uses Kubernetes containers, we'll see containers OK. However, if they use something else (e.g. docker containers) then it won't show?

Example if docker containers were used

Inventory : Docker Containers

In the inventory UI, we only support docker containers (not Kubernetes containers):

Desired State

We'd like to be able to show containers (regardless of K8s/docker or whatever) in both Inventory and the Infrastructure Tab in APM.

Therefore, completing this ticket will allow for this?

My thoughts:

I'd love to invest in doing a really good job with containers in both the APM UI and Inventory by setting some time aside to really figure this out.

I'm wondering whether we could perhaps tackle this use case as part of a wider epic on how containers should show around the UI?

e.g. We could have potentially have containers showing up within a host detail/fly-out view (certainly for K8s within pods) - perhaps we could get them handled as assets?

@smith / @neptunian - Not urgent but do you have any thoughts on how it might make sense to tackle containers on our roadmap? If completing this ticket would be a quick fix for the customer issue raised then I think there's a good argument to trying to do this sooner (rather than later). However, if this work wouldn't be beneficial when we get to an asset-centric world then I think we should defer this and try and do a great job of it later.

[neptunian]

If a user uses Kubernetes containers, we'll see containers OK. However, if they use something else (e.g. docker containers) then it won't show?

If they use the kubernetes module, they'll get the metrics. If they are using the docker module, we are not using those metrics in the table.

In the inventory UI, we only support docker containers (not Kubernetes containers):

Looks like some kubernetes container metrics are now supported as well with https://github.com/elastic/kibana/issues/134467

Not urgent but do you have any thoughts on how it might make sense to tackle containers on our roadmap? If completing this ticket would be a quick fix for the customer issue raised then I think there's a good argument to trying to do this sooner (rather than later).

Probably an investigative issue that comes up with a plan as the AC states. Should consider otel metrics as well. This is not a "quick fix" situation, though.

However, if this work wouldn't be beneficial when we get to an asset-centric world then I think we should defer this and try and do a great job of it later.

In the asset centric world we'd still have to deal with various metrics

[roshan-elastic]

Completely missed this @neptunian - sorry for the delay.

@smith / @neptunian - What do you think about an epic to start tackling 'containers' in our roadmap? Perhaps some discovery to help us figure out how we want to show these in our UI?

Examples

• Showing containers running within a host
• Showing containers running within other assets (e.g. pod)
• Showing containers within inventory
• Showing containers within APM 'infrastructure' tab

[smith]

@roshan-elastic +1 on containers in the UI.

[roshan-elastic]

Cool - I created this epic in the backlog for discovery later:

• [ ] https://github.com/elastic/obs-infraobs-team/issues/1198

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[botelastic[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[roshan-elastic]

Closing as we've either solved part of this already or are planning to fix this in the near future.

We'll refer to this issue in the work that we do to ensure we don't miss this.