Missing request timeout config for ruler_storage

[KaiserNicols]

Describe the bug Observing timeouts when the Ruler makes a request to Azure storage (/api/v1/rules) to fetch the alert rules. We have roughly 60k rules in storage for a particular tenant, so could be a very large payload if everything comes in a single response.

I suspect the default request timeout is 30s, but I don't see a 'request_timeout' field for Azure in the ruler_storage config.

#4581 added some http configs for Azure storage, but not for a request timeout. I no longer see this in the Cortex docs, but previously there was ruler.storage.azure.request_timeout. This setting is still visible via the /config api.

I get this timeout in Postman as well.

To Reproduce Run Cortex

• Distributed mode on AKS, rendered with Helm
• 1.13.0

Connect the ruler to an azure storage (via the ruler_storage config). Upload a large amount of rules to storage Try to have ruler fetch the rules (I believe this is done via /api/v1/rules)

Expected behavior Ruler should fetch the alerts, and not get timed out.

[alvinlin123]

Hello, can you try the ruler-storage.azure.http.response-header-timeout to see if that works for you?

The ruler.storage.* config have been deprecated for a while, and is being removed from Cortex, that's why you don't see the config on the Cortex website now. ruler.storage.* is still supported in 1.13.0, but please migrate to using ruler_storage configuration instead.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.