Carson Ip

Observation: when `monitoring.enabled: true` and `monitoring.elasticsearch` is configured, the metrics are sent to ES directly under `.monitoring-beats-7-*`. The `output.elasticsearch.*` fields exist in source but are unmapped.

In that case there may be 2 problems: 1. the `.monitoring-beats-7-*` does not have the mappings for some of our fields 2. metricbeat beat module may be ignoring the fields...

> metricbeat beat module may be ignoring the fields because some of our fields do not fall into ["exported fields" defined by metricbeat](https://www.elastic.co/guide/en/beats/metricbeat/8.14/exported-fields-beat.html). Pending confirmation. (2) is not right. The...

It should be closed by https://github.com/elastic/apm-server/issues/13475

This is less straightforward than before. In 8.x there is one configured storage limit, and we compare db size against that. However, from 9.0, by default the storage limit is...

`apm-server.sampling.tail.events.failed_writes` is a counter metric that records the number of failed writes, no matter it is then discarded or directly indexed to ES. If this per-apm-server counter increases, it means...

That said, for https://github.com/elastic/kibana/issues/226600 , I wonder how kibana would have access to monitoring metrics (think about stack monitoring). If we cannot surface this by default on ECH, I wonder...

In case we move ahead with implementing this, the actual work involves adding this metric in apm-server code, and update relevant mappings in ES, integrations and metricbeat repo. See https://github.com/elastic/apm-server/issues/13475...

As discussed during weekly, we would like to expand the scope of this task and ensure that it is relevant and valuable to both 8.x and 9.x. The description has...

I'll bring this to the team and see if we want to bump go-elasticsearch to v9 at all, because IIUC it has some backward / forward compatibility implications.