cortex icon indicating copy to clipboard operation
cortex copied to clipboard

Update remote read to the new streaming interface.

Open tomwilkie opened this issue 5 years ago • 4 comments

@bwplotka introduced a new streaming remote read in Prometheus, we should add the same to Cortex for people who point Prometheus and Thanos at Cortex.

tomwilkie avatar Apr 21 '20 14:04 tomwilkie

@tomwilkie I'd like to start working on this issue. Can you or @bwplotka point me in the right direction to read about that new streaming remote read?

Thx!

ethervoid avatar May 07 '21 11:05 ethervoid

Hi @ethervoid - thank you!

First thing to look at is the existing remote read implementation in Cortex, which can be found here: https://github.com/cortexproject/cortex/blob/master/pkg/querier/remote_read.go

Next, checkout the new prometheus streaming remote read, which I actually refactored into a separate "module" here: https://github.com/prometheus/prometheus/pull/8536. This should make it pretty easy to reuse, but you might need to make some changes upstream.

Finally, it will be useful to have a local test environment, so take a look at our gettinig started guide: https://cortexmetrics.io/docs/getting-started/. Try and extend this to get the Prometheus instance (or another instance) remote reading back from Cortex.

Let me know if you want a quick chat about this.

tomwilkie avatar May 07 '21 18:05 tomwilkie

@tomwilkie Thank you very much. I'll take a look at this and when I have a clear picture I'd love to have a chat about it :)

ethervoid avatar May 10 '21 08:05 ethervoid

@tomwilkie I'm working on it today, can you assign it to me :)

ethervoid avatar May 14 '21 15:05 ethervoid