Bryan Boreham

Also document whether remote-write is supposed to send empty external labels, because I'm pretty sure Cortex and Mimir are broken if you do that.

My first quote is from “Data Model”; no mention of scraping or emitting. Seriously, I don’t understand what you meant about federation and honor_label; I quoted what I could find...

OK I think what you are describing is: * The client side of `/federate` is normal scraping. * Scraping adds labels like `job` and `instance`. Federation docs tell you to...

@prymitive thanks for that; somehow the buffer it's trying to use is too small. Anything you can tell me about the query being executed when it crashed?

> Prometheus never crashed, panic was recovered. That seems like a good opportunity for improvement. If a panic reaches the http handler then log the query. Separately, can you say...

> I see a few panics in logs. New theory: the problem is concurrent use of [this `SimpleBuilder`](https://github.com/prometheus/prometheus/pull/10991/files#diff-fb0f63e7d2d24fa7aa1c18b2619b10a07c821b38f45c2351458b00edef63f995R1781). The size of the buffer is calculated with one set of labels,...

I managed to recreate the race I theorised in previous message in a unit test, and moved the state into `blockBaseSeriesSet` which makes the report go away. Unfortunately the new...

> Somehow `external_labels` are missing from `/api/v1/status/config` response, as if marshaling to JSON didn't work properly. I fixed a flaw with marshalling to YAML, which I think you meant. Also...

Thanks; I had not thought of dictionary-style compression. I guess the decompression would happen in `Range()` etc.? Couple of ideas I had considered, as small modifications to this current PR:...

I have what looks like the same failure on GitHub Actions: https://github.com/prometheus/prometheus/runs/8251749473