Missing option to disable traefik's reverse proxy always appending the RemoteAddr to `x-forwarded-for` header

[jetersen]

Welcome!

• [x] Yes, I've searched similar issues on GitHub and didn't find any.
• [x] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?

Currently trying to migrate from ingress nginx to traefik.

Our setup is this: Client > AWS Cloudfront > AWS NLB > Traefik > Pod

ingress nginx has a config option called ComputeFullForwardedFor by default in ingress nginx it is set to false but if you enable it behaves how traefik currently behaves by always appending the RemoteAddr to x-forwarded-for header.

However all our application are coded only read 1 IP from x-forwarded-for header and since it is read from right to left it happens to read the cloud provider load balancer ip because traefik appended it and I would like to avoid changing all our applications.

I would like traefik to have an option to not append RemoteAddr so either Middleware can calculate the IP or trust the cloud provider either via it's x-forwarded-for or proxy protocol.

I created this plugin to change the x-forwarded-for and read it from Cloudfront-Viewer-Address.

Basically I am looking for preserving the x-forwarded-for either from loadbalancer or middleware without the traefik reverse proxy middleware appending RemoteAddr.

https://github.com/traefik/traefik/blob/c20802b07ea8efd6638e0d126eec5df6d932f58f/pkg/proxy/fast/proxy.go#L215-L228

Considering NGINX Ingress Provider was added this makes sense to support this behavior to allow for smoother migration.

[shreealt]

this sounds like a legit use case to me, if the maintainers agree I would be happy to pick this up as I have solved similar problems before :D (it's an interesting, yet common problem)

[kevinpollet]

Hello @jetersen and thanks for opening this,

Today, Traefik appends the previous hop IP (RemoteAddr) to the x-forwarded-for header when the previous hop is trusted (as specified here). When the previous hop is not trusted, Traefik will overwrite the header value by setting it to the previous hop IP.

Please could you describe the expected behavior and which IP you want in the x-forwarded-for header?

[jetersen]

Well I trust my VPC cidr but I do not want it to add the load balancer to the x-forwarded-for if I don't trust my VPC I get a bad request. For the x-forwarded-forI want a singular IP that traefik will set based on either Cloudfront-Viewer-Addressor via proxy protocol.

~I am using proxy protocol maybe that's the reason I get bad request.~ The x-forwarded-ip is still the NLB ip after setting the trustedIPs to 1.1.1.1/32 when using proxy protocol and no x-forwarded-for is passed in.

Provided example and config below of the cloudfront plugin I wrote picking up the Cloudfront-Viewer-Address and setting it as the X-Forwarded-For but traefik seems to always appends the 10.0.1.99 because of the default behavior. No matter if the CIDR is trusted or not. If I trusted the proxy-protocol from the VPC I would see Cloudfront RemoteAddr ip.

This is the config option from ingress-nginx I referred to

compute-full-forwarded-for Append the remote address to the X-Forwarded-For header instead of replacing it. When this option is enabled, the upstream application is responsible for extracting the client IP based on its own list of trusted proxies.

So when it is disabled in ingress nginx it replaces the x-forwarded with whatever the realip module header determines to be the correct ip.

The desired effect is that Traefik is responsible for extracting the correct IP and replace it in the x-forwarded-for header as the upstream service only cares about the extracted ip.

But currently this is the result:

    "X-Forwarded-For": [
      "108.128.180.187:1024, 10.0.1.99"
    ],

This is what I desire:

    "X-Forwarded-For": [
      "108.128.180.187:1024"
    ],

  service:
    spec:
      externalTrafficPolicy: Local
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-name: traefik-internal
      service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
      service.beta.kubernetes.io/aws-load-balancer-type: external
      service.beta.kubernetes.io/aws-load-balancer-scheme: internal
      service.beta.kubernetes.io/aws-load-balancer-alpn-policy: HTTP2Optional
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: websecure
      service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
      service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: proxy_protocol_v2.enabled=true
  experimental:
    plugins:
      cloudfrontxforwarded:
        moduleName: github.com/jetersen/traefik-cloudfront-xforwarded
        version: v0.1.0
  ports:
    web:
      middlewares:
        - traefik-cloudfrontxforwarded@kubernetescrd
      forwardedHeaders:
        trustedIPs:
          - 1.1.1.1/32
      proxyProtocol:
        trustedIPs:
          - 1.1.1.1/32
    websecure:
      forwardedHeaders:
        trustedIPs:
          - 1.1.1.1/32
      proxyProtocol:
        trustedIPs:
          - 1.1.1.1/32

[rtribotte]

@jetersen Thank you for your reply!

Is it correct to summarize this issue as a request for a way to prevent Traefik from systematically appending the remote address to the X-Forwarded-For header?

[jetersen]

@rtribotte Yes. That is exactly what the title of the issue is saying.

updated title to clarify the header x-forwarded-for header part.

[nmengin]

Hey @jetersen.

Thanks for the additional information.

We are interested in this issue, but we’re unsure about the use case and the traction it will receive. We are going to leave the status as kind/proposal to give the community time to let us know if they would like this idea. We will reevaluate as people respond. Conversation is time-boxed to 6 months.

[jetersen]

so ingress nginx has the option and so does envoy gateway/proxy as skip-xff-append

[si-c613]

Just started working on a middleware for the Cloudflare headers and have also come across this problem. We currently use Nginx which has built in config to dictate which header to use as the forwarded for headers and real ip.

Same problem as above, just a different CDN.

[rtribotte]

Hey @jetersen @si-c613,

We have marked this issue as an enhancement and put it on our (maintainer) plate.

[jetersen]

@rtribotte excellent to hear