Allow Raw HTTP input to understand x-forwarded-for or x-real-ip headers

[drewmiranda-gl]

What?

Modify the Graylog Raw HTTP input to read/understand the following headers sent by load balancers:

• x-forwarded-for
• x-real-ip

And use these to set the source field if no other suitable fields are present.

Why?

Customers may need or want to put a load balancer in front of Graylog's Raw HTTP input for redundancy and high availability. However, when web traffic is proxied through a load balancer, lets say HA Proxy, the true log source is obscured.

Load Balancers solve for this using standard headers such as

• x-forwarded-for (original hostname of source)
• x-real-ip (original IP address of source)

This allows the Graylog Raw HTTP input to properly set the true log source. The log source is very important, especially in audit log scenarios where it is critical to identify what server or log source sent the log message.

This was originally requested by a customer.

Please let me know if there are any questions.

[boosty]

X-Forwarded-For can contain multiple entries, and extra care needs to be taken to extract the real IP (e.g., it should be allowed to configure a list of known proxies): https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Forwarded-For#security_and_privacy_concerns

Another related header exists, but it does not seem to be very popular: Forwarded https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Forwarded

To our users I would strongly suggest to store the raw values of these headers as extra fields for later inspection, even when this feature would be implemented.