Support to prometheus native histogram

[jeromeinsf]

Is your feature request related to a problem? Please describe. Experimental native histogram support was released with Prometheus v0.40.x.

Describe the solution you'd like We should implement native histogram support (behind a feature flag)

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

[amouhadi]

just checking if there is any plan for supporting prometheus native histogram great news opentelemetry collector now supports the conversion as well https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/17370

[friedrichg]

@amouhadi yes that is the plan

[friedrichg]

https://github.com/cortexproject/cortex/pull/5289 this tracks how many samples for this histogram is cortex dropping. It's a start

[hkfgo]

Hey just checking in again to see if anything's changed. Is native histogram support still on the roadmap?

[yeya24]

Yeah I believe it is still on the roadmap. We need to wait our upstream project Thanos to support native histograms first.

[yeya24]

If I understand correctly, Thanos already supports native histogram on all components other than downsampling. We should start integrating with native histograms on Cortex

[yeya24]

Let me update the checklist for steps to enable native histograms. Feel free to leave comments about missing items.

Checklist

• [x] Distributor
  • Distributor should send native histograms in its write request https://github.com/cortexproject/cortex/pull/5679 #6020
• [x] Ingester
  • Ingester is able to ingest native histograms to TSDB https://github.com/cortexproject/cortex/pull/5986 #6010
• [x] Query Frontend
  • Include native histogram as part of Query response, merge native histogram samples correctly https://github.com/cortexproject/cortex/pull/5996
  • Support for native histogram in instant queries #6043
• [x] Querier
  • Prerequisite change and cleanup: #5694
  • Allow querier to be able to handle native histograms from Store Gateway: #5884
  • Change querier series interface to include native histogram chunks https://github.com/cortexproject/cortex/pull/5944
  • More tests https://github.com/cortexproject/cortex/pull/6031
• [x] Ruler
  • Ingest native histograms to ingester https://github.com/cortexproject/cortex/pull/6029
• [x] Store Gateway
  • Make sure Store Gateway can return native histogram in response (should work as Thanos supports it)
• [x] Compactor
  • Make sure compactor compacts native histogram series (should work once TSDB up-to-date)

[gburton1]

How's the checklist above looking? Any recent updates?

[hkfgo]

I see an August date for native histogram support, under the cortex project page. Is that date an estimate?

[yeya24]

Thanks. I updated the checklist above. I think the ingestion path is done. Query path are almost done. Things left:

• Ruler
• Querier queries histogram iterator when asked for float histogram

Also some more tests need to be added. I expect it to be released as part of 1.18 release.

[yeya24]

With all the PRs merged, I think we can mark this issue as resolved.

I will create another issue to track any efforts post merge.

Updated: new tracking issue https://github.com/cortexproject/cortex/issues/6052