loki icon indicating copy to clipboard operation
loki copied to clipboard

feat(helm): add location snippet to nginx config

Open joschi36 opened this issue 5 months ago • 2 comments

What this PR does / why we need it:

The current Helm chart for the Loki gateway does not provide a way to inject Nginx configuration directives inside individual location blocks. This makes it impossible to implement common and important authentication or header manipulation schemes that rely on directives like proxy_set_header without overriding the entire Nginx configuration file.

A key use case is mTLS-based multi-tenancy, where an X-Scope-OrgID header must be set based on client certificate details. Due to Nginx's directive inheritance rules, setting this header in serverSnippet is overridden by any location block that defines its own proxy_set_header (e.g., for X-Query-Tags in the /loki/api/v1/ location or WebSocket upgrades in the /loki/api/v1/tail location). This results in failed authentication for some endpoints but not others.

Which issue(s) this PR fixes: Fixes #

Special notes for your reviewer:

This change is fully backward-compatible. If locationSnippet is not set, the template renders exactly as it did before, resulting in no change for existing users.

This was tested by implementing the mTLS multi-tenancy scheme described above, which was previously only possible via complex workarounds like overriding the entire nginx.conf file or using a post-renderer. With this change, the configuration becomes trivial and is managed cleanly through values.yaml. The snippet has been added to all location blocks for consistency and to ensure all API endpoints served by the gateway are covered.

Checklist

  • [x] Reviewed the CONTRIBUTING.md guide (required)
  • [ ] Documentation added
  • [ ] Tests updated
  • [x] Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • [ ] Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • [ ] If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

joschi36 avatar Jun 13 '25 16:06 joschi36

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Jun 13 '25 16:06 CLAassistant

Seems to be like a duplicate of #11348

Can we go forward with either one? Customizing the clientMaxBodySize would benefit us as well

joschi36 avatar Jun 16 '25 15:06 joschi36

Hi @jkroepke

Could you also please provide an use-case for this?

I have provided a use case in the PR description. Do you mean adding it in the values.yaml as an example use case?

A key use case is mTLS-based multi-tenancy, where an X-Scope-OrgID header must be set based on client certificate details. Due to Nginx's directive inheritance rules, setting this header in serverSnippet is overridden by any location block that defines its own proxy_set_header (e.g., for X-Query-Tags in the /loki/api/v1/ location or WebSocket upgrades in the /loki/api/v1/tail location). This results in failed authentication for some endpoints but not others.

Or do you mean regarding clientMaxBodySize. I basically have the same as @panzouh has in his initial PR #11348

Adjusting client_max_body_size in Nginx could allow me fixing nginx errors client intended to send too large body.

But I didn't know until now that this was already solved: https://github.com/grafana/loki/commit/809a024581c1f600744b9db0b2b2142234317082

joschi36 avatar Jul 09 '25 11:07 joschi36

LGTM

Thanks for your answers.

jkroepke avatar Jul 09 '25 13:07 jkroepke

add a note to changelog, including PR at the end of the note.

yes just noticed #18414 aswell. This would indeed solve my issue. do we still want to merge this one?

nevertheless I've updated the branch, implemented you proposed change and added the changelog

joschi36 avatar Jul 15 '25 09:07 joschi36

LGTM.

I didnt have merge power yet, but one of the internal loki team will merge this soon.

jkroepke avatar Jul 15 '25 09:07 jkroepke

@joschi36 please check CI looks like make helm-docs is missing

jkroepke avatar Jul 18 '25 17:07 jkroepke

thx @nicolevanderhoeven

joschi36 avatar Jul 22 '25 09:07 joschi36

Generated the reference doc to unblock this and resolved a merge conflict with Changelog. :) Just waiting for final checks, and otherwise LGTM and I'll merge!

nicolevanderhoeven avatar Jul 22 '25 09:07 nicolevanderhoeven