apm-server Enable synthetic source for metrics data streams

We should investigate enabling synthetic source for metrics data streams.

Metric documents are typically only used in aggregations, with _source being irrelevant. We can save on storage space by synthesising source on-demand, for the exceptional cases where users wish to inspect the source in Discover.

Enabling synthetic source may require https://github.com/elastic/elasticsearch/pull/89466, as Fleet automatically adds ignore_above to keyword fields.

Aug 31 '22 02:08 axw

related: https://github.com/elastic/package-spec/issues/340

Aug 31 '22 05:08 simitt

https://github.com/elastic/elasticsearch/pull/89466 has been merged, but unfortunately we still can't enable synthetic source:

2022/09/06 18:20:08 illegal_argument_exception: [illegal_argument_exception] Reason: field [transaction.duration.histogram] of type [histogram] doesn't support synthetic source

Sep 06 '22 10:09 axw

Should be unblocked soon :crossed_fingers: https://github.com/elastic/elasticsearch/pull/89833

Sep 07 '22 02:09 axw

We'll keep looking at this during 8.5 development, but won't actually turn it on by default until 8.6 - retargeting this.

Sep 09 '22 05:09 axw