Bug: downstream replied with Unprocessable Entity

[lucasfnds]

Describe the bug

In Grafana, when trying to use Mimir's Datasource, the error appears on the screen: "Validation failed too many unhealthy instances in the ring". When analyzing the query-frontend logs we can notice:

`ts=2024-08-13T13:47:42.036133953Z caller=handler.go:372 level=info user=anonymous msg="query stats" component=query-frontend method=GET path=/prometheus/api/v1/label/name/values user_agent=Grafana/11.1.3 status_code=422 response_time=2.701379ms response_size_bytes=123 query_wall_time_seconds=0.000694019 fetched_series_count=0 fetched_chunk_bytes=0 fetched_chunks_count=0 fetched_index_bytes=0 sharded_queries=0 split_queries=0 estimated_series_count=0 queue_time_seconds=2.8571e-05 param_start=2024-08-13T12:47:00Z param_end=2024-08-13T13:48:00Z length=1h1m0s time_since_min_time=1h0m42.033364973s time_since_max_time=-17.966635027s results_cache_hit_bytes=0 results_cache_miss_bytes=0 status=failed err="downstream replied with Unprocessable Entity"

ts=2024-08-13T13:47:42.19402017Z caller=handler.go:372 level=info user=anonymous msg="query stats" component=query-frontend method=POST path=/prometheus/api/v1/labels user_agent=Grafana/11.1.3 status_code=422 response_time=3.283817ms response_size_bytes=123 query_wall_time_seconds=0.00154828 fetched_series_count=0 fetched_chunk_bytes=0 fetched_chunks_count=0 fetched_index_bytes=0 sharded_queries=0 split_queries=0 estimated_series_count=0 queue_time_seconds=2.483e-05 param_start=2024-08-13T12:47:00Z param_end=2024-08-13T13:48:00Z length=1h1m0s time_since_min_time=1h0m42.190677582s time_since_max_time=-17.809322418s results_cache_hit_bytes=0 results_cache_miss_bytes=0 status=failed err="downstream replied with Unprocessable Entity"`

I don't have any description of what could be happening, the logs didn't help me.

To Reproduce

Steps to reproduce the behavior:

1. Install grafana mimir using helm;
2. Add the Prometheus Datasource to Grafana pointing to Mimir with Prometheus type" = "Mimir" and save;
3. Click on "Explore view";

Environment

• Infrastructure: Kubernetes
• Deployment tool: Helm
• Mimir helm version: 5.4.0
• Grafana version: 11.1.3