CloudFlare mod_remoteip

[dnburgess]

When looking throug my logs, I see that all the IPs shown are from CloudFlare, which makes sense as I am using CloudFlare for DNS and security.

Do you have any plans to allow for mod_remoteip so that we can see the real IP addresses of the people accessing our containers rather than the CloudFlare IP addresses?

[xavier-hernandez]

I'm not familiar with that implementation. I'll set up a test environment on my end and review it. Do you know if the client IP is in the logs?

[dnburgess]

The client IP is not in the logs as the IPs are proxied from CloudFlare unless you use mod_remoteip

[xavier-hernandez]

If you use mod_remoteip are the log files created differently. If so, can you send me a sample log.

I was reviewing documentation for NginxProxyManager and it seems that the real IP is loaded into the logs rather than the Cloudflare IPs so it should work out of the box. https://github.com/NginxProxyManager/nginx-proxy-manager/issues/919#issuecomment-1025171063

[dnburgess]

I'm honestly not sure, but in order to get Fail2Ban to work with Nginx Proxy Manager (NPM), I had to go through a bit of setup. You can see the process I went through in this video: https://www.youtube.com/watch?v=Ha8NIAOsNvo

Or you can see the write-up I did for that video in order to get NPM/Fail2Ban to work with CloudFlare to get the real IPs: https://dbt3ch.com/books/fail2ban/page/how-to-install-and-configure-fail2ban-to-work-with-nginx-proxy-manager

When I look at the "Visitor Hostnames and IPs" on the goaccess dashboard, I see approximately 50 Ip addresses

Here is a snippet of the logs from NPM: logs.txt

Somehow the fail2ban container (as far as I can tell) connects to CloudFlare via the API and gets the real information, then communicates with CloudFlare to ban the appropriate IP address

Here is a screenshot of my CloudFlare dashboard showing 400+ IPs banned from my fail2ban setup that is configured with NPM

[xavier-hernandez]

Still reviewing this.

[itsKV]

Any progress here?

[xavier-hernandez]

@itsKV I have been looking into it and I don't think it's possible in the way the docker image currently runs and the way logs are handled in general. GoAccess does not currently support this functionality. Its only real job is to parse whatever logs you provide it. GOAN, what I'm calling this docker image now, is just really a helper to get the logs from NPM to GoAccess in a friendly manner without the user needing to do a lot of setups.

As GOAN is in the middle of NPM and GoAccess I am able to do some things but nothing that would remain persistent through restarts. I can and would be able to read the logs as they come in real-time or not and query Cloudflare via API for the real IP however processing that data might take a long time. For example, I currently have a million requests in my log. I would need to query Cloudflare a million times. Data would not be shown on the screen for some time. Then if the container gets restarted the whole process starts over again.

The manipulation I think needs to take place on the NPM side. This way the right IP is written to the log and GoAccess and GOAN just parse and display the data for you. Reference: https://github.com/NginxProxyManager/nginx-proxy-manager/issues/811. I have not tried it myself but it might work.

Other options I've thought about.

• Look at GoAccess ability to store data in a persistent way and manipulate that data.
• Only convert data that comes in once GOAN is up and running, but this would leave the majority of the data not touched and on restart you'd lose the data already converted.
• Create another project and retrieve logs from Cloudflare and have GoAccess parse them. But would that provide any additional data that Cloudflare doesn't show on their dashboard already?

Let me know if you think of anything else. I'm not an expert so recommendations or ideas are appreciated.

[calebpaine]

I'll check this later myself, but it should be fixable by adding real_ip_header CF-Connecting-IP; to the Advanced config of each proxy configured in NPM. This fixes my IP-based ACL issues within NPM, so I assume it also fixes the logging.

[xavier-hernandez]

@calebpaine something like that should work. Modifications can be done on the NPM side to translate the IPs but not on my container end.