wp-rocket icon indicating copy to clipboard operation
wp-rocket copied to clipboard

Varnish Purge on Cloudways

Open Daniyal-DigitalOcean opened this issue 3 weeks ago • 17 comments

Describe the bug I am Senior Engineer from the Cloudways reporting an important issue which was recently observed with the wp-rocket and varnish purging. Cloudways has recently change their VCL where PURGE request will only work when the Real-IP header and request and is routed through the Nginx but on the Wp-rocket, The request is directly going on the varnish without the real IP header and PURGE request is getting blocked.

Working PURGE request:

*   << Request  >> 65581     
-   Begin          req 65580 rxreq
-   Timestamp      Start: 1764849175.476370 0.000000 0.000000
-   Timestamp      Req: 1764849175.476370 0.000000 0.000000
-   ReqStart       127.0.0.1 15930 a0
-   ReqMethod      PURGE
-   ReqURL         /.*
-   ReqProtocol    HTTP/1.0
-   ReqHeader      X-Real-IP: 127.0.0.1
-   ReqHeader      X-Forwarded-For: 127.0.0.1
-   ReqHeader      X-Client-Real-IP: 127.0.0.1
-   ReqHeader      Host: wordpress-1559331-6048096.cloudwaysapps.com
-   ReqHeader      X-Forwarded-Proto: http
-   ReqHeader      X-Forwarded-Host: wordpress-1559331-6048096.cloudwaysapps.com
-   ReqHeader      X-Application: wordpress
-   ReqHeader      X-App-User: vjjvvmutve
-   ReqHeader      X-Version: latest
-   ReqHeader      Connection: close
-   ReqHeader      User-Agent: W3 Total Cache
-   ReqUnset       X-Forwarded-For: 127.0.0.1
-   ReqHeader      X-Forwarded-For: 127.0.0.1, 127.0.0.1
-   VCL_call       RECV
-   ReqUnset       X-Forwarded-For: 127.0.0.1, 127.0.0.1
-   ReqHeader      x-forwarded-for: 127.0.0.1
-   ReqUnset       Host: wordpress-xxxx-xxxx.cloudwaysapps.com
-   ReqHeader      Host: wordpress-xxxx-xxxx.cloudwaysapps.com
-   VCL_return     purge
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       PURGE
-   VCL_return     synth
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespReason     Purged
-   RespHeader     Date: Thu, 04 Dec 2025 11:52:55 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 65581
-   VCL_call       SYNTH
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   VCL_return     deliver
-   Timestamp      Process: 1764849175.476509 0.000139 0.000139
-   RespHeader     Content-Length: 0
-   Storage        malloc Transient
-   RespHeader     Accept-Ranges: bytes
-   RespHeader     Connection: close
-   Timestamp      Resp: 1764849175.476591 0.000221 0.000083
-   ReqAcct        356 0 356 211 0 211
-   End            

Request through WP-Rocket:

Begin          req 65573 rxreq
Timestamp      Start: 1764842237.777350 0.000000 0.000000
Timestamp      Req: 1764842237.777350 0.000000 0.000000
ReqStart       127.0.0.1 53716 a0
ReqMethod      PURGE
ReqURL         /page/.*
ReqProtocol    HTTP/1.1
ReqHeader      Host: http://wordpress-xxx-xxx.cloudwaysapps.com 
ReqHeader      User-Agent: WordPress/6.9; https://wordpress-xxx-xxx.cloudwaysapps.com 
ReqHeader      Accept: /
ReqHeader      Accept-Encoding: deflate, gzip, br, zstd
ReqHeader      X-Purge-Method: regex
ReqHeader      Connection: close
ReqHeader      X-Forwarded-For: 127.0.0.1
VCL_call       RECV
ReqUnset       X-Forwarded-For: 127.0.0.1
ReqHeader      x-forwarded-for:
VCL_return     synth
ReqUnset       Accept-Encoding: deflate, gzip, br, zstd
ReqHeader      Accept-Encoding: gzip
VCL_call       HASH
VCL_return     lookup
RespProtocol   HTTP/1.1
RespStatus     405
RespReason     Method Not Allowed
RespReason     This IP is not allowed to send PURGE requests.
RespHeader     Date: Thu, 04 Dec 2025 09:57:17 GMT
RespHeader     Server: Varnish
RespHeader     X-Varnish: 65574
VCL_call       SYNTH
RespHeader     Content-Type: text/html; charset=utf-8
RespHeader     Retry-After: 5
VCL_return     deliver
Timestamp      Process: 1764842237.777404 0.000054 0.000054
RespHeader     Content-Length: 0
Storage        malloc Transient
RespHeader     Connection: close
Timestamp      Resp: 1764842237.777443 0.000093 0.000039
ReqAcct        255 0 255 229 0 229
End

The reverse proxy configuration for Varnish should be set through port 80 so that all requests are routed through Nginx, ensuring the request headers—such as the real-ip header—are preserved as expected. Without this adjustment, thousands of users will be unable to use WP-Rocket on the new Cloudways servers. We also intend to apply this change to the older servers, so please prioritize and expedite this ticket to prevent any plugin compatibility issues.

Varnish VCL:

# PURGING
# SMART PURGE CACHE BY URL
if (req.method == "URLPURGE") {
        if (req.http.X-Real-IP != "127.0.0.1" && req.http.X-Real-IP != "::1" && req.http.X-Real-IP != "localhost" && req.http.X-Real-IP != "157.245.126.171") {
                return(synth(405, "This IP is not allowed to send PURGE requests."));
        }
        ban("req.http.host == " + req.http.host + " && req.url == " + req.url);
        return(synth(200, "Purged"));
}

# PURGE CACHE BY HOSTNAME
if (req.method == "PURGE") {
        if (req.http.X-Real-IP != "127.0.0.1" && req.http.X-Real-IP != "::1" && req.http.X-Real-IP != "localhost" && req.http.X-Real-IP != "157.245.126.171") {
                return(synth(405, "Not allowed."));
        }
        ban("req.http.host ~ " + req.http.host);
        return (purge);
}

#BANING
if (req.method == "BAN") {
        if (req.http.X-Real-IP != "127.0.0.1" && req.http.X-Real-IP != "::1" && req.http.X-Real-IP != "localhost" && req.http.X-Real-IP != "157.245.126.171") {
                return(synth(405, "This IP is not allowed to send PURGE requests."));
        }
        ban("req.http.host == " + req.http.host + " && req.url == " + req.url);
        return(synth(200, "Ban added"));
}

Daniyal-DigitalOcean avatar Dec 04 '25 13:12 Daniyal-DigitalOcean

@DahmaniAdame @piotrbak Can we please look into it on priority?

cwanasmoiz avatar Dec 08 '25 07:12 cwanasmoiz

@engahmeds3ed would you mind looking into this?

AC Varnish purge should be compatible with Cloudways requirements.

DahmaniAdame avatar Dec 08 '25 07:12 DahmaniAdame

Context

Cloudways recently updated their Varnish configuration and now our cache purge is failing. Their engineering team reported that we're sending PURGE requests directly to port 8080 (Varnish) instead of routing through Nginx on port 80, which means the X-Real-IP header is missing from our requests.

Looking at the Varnish logs they shared, W3 Total Cache works fine because their requests include the proper headers and go through port 80. Our requests are getting blocked with a 405 error.

What's Happening

I traced this back to our Cloudways integration in inc/ThirdParty/Hostings/Cloudways.php. Line 114 explicitly sets the Varnish IP to 127.0.0.1:8080, which bypasses their Nginx reverse proxy.

Their new VCL requires the X-Real-IP header to validate PURGE requests. When we go directly to port 8080, Nginx never touches the request, so that header is never added. This is breaking cache purging for their users.

Here's what their logs show:

Our failing request:

ReqMethod      PURGE
ReqURL         /page/.*
ReqHeader      Host: http://wordpress-xxx-xxx.cloudwaysapps.com 
# No X-Real-IP header
VCL_return     synth
RespStatus     405
RespReason     This IP is not allowed to send PURGE requests.

W3TC's working request:

ReqMethod      PURGE
ReqURL         /.*
ReqHeader      X-Real-IP: 127.0.0.1
ReqHeader      X-Forwarded-For: 127.0.0.1
ReqHeader      X-Client-Real-IP: 127.0.0.1
VCL_return     purge
RespStatus     200
RespReason     Purged

Scope a solution

What Needs to Change

We need to route PURGE requests through port 80 instead of 8080. The question is whether we should just change the port and let Nginx handle the headers, or if we should also explicitly add the X-Real-IP header ourselves.

I'm leaning towards doing both - change the port AND add the header. Looking at how we handle O2Switch in inc/ThirdParty/Hostings/O2Switch.php, they do something similar by hooking into rocket_varnish_purge_headers to modify the request headers. This seems like a safer approach since we're not relying solely on Nginx's configuration.

Things to Consider

The main file to update is inc/ThirdParty/Hostings/Cloudways.php. We'd need to:

  • Change the port in the varnish_ip() method
  • Hook into the rocket_varnish_purge_headers filter to add the X-Real-IP header
  • Make sure this only happens when Varnish is actually running (we already have the detection logic)

For testing, we'll need to update the existing test fixtures since they all expect port 8080. Plus we should add tests for the header injection to make sure it works correctly and doesn't break anything else.

References

  • Similar implementation in O2Switch: inc/ThirdParty/Hostings/O2Switch.php
  • Varnish core logic: inc/Addon/Varnish/Varnish.php
  • Existing tests: tests/Unit/inc/ThirdParty/Hostings/Cloudways/ and tests/Integration/inc/ThirdParty/Hostings/Cloudways/

Miraeld avatar Dec 08 '25 09:12 Miraeld

I'm thinking we will not need to add the X-Real-IP header, I believe cloudways pre configure this it just disappears when we use port 8080, any ways we'll need to test this during development.

LGTM.

jeawhanlee avatar Dec 08 '25 11:12 jeawhanlee

@Daniyal-DigitalOcean Can u plz confirm the above comment, if we need to add X-Real-IP header when we change the port to be 80 instead of 8080 ?

wordpressfan avatar Dec 09 '25 10:12 wordpressfan

@wordpressfan Yes, Nginx automatically adds the X-Real-IP header, so there’s no need to send it manually.

cwanasmoiz avatar Dec 09 '25 11:12 cwanasmoiz

@cwanasmoiz Thanks for your help so far, we have the PR ready but we couldn't find an easy way to test this fix. If you don't mind, could you recommend ways to test this. We have a cloudways testing site. I believe we just need to assert that X-Real-IP is part of the req header and we get a response status of 200. Is it possible that we perform a cache clear there with WP Rocket and you send us a log just like in the issue or we attach a compiled version of WP Rocket here with this fix for you to test. I believe this would be helpful. Which do you prefer?

jeawhanlee avatar Dec 10 '25 08:12 jeawhanlee

Hi @jeawhanlee, It will be better if you could provide us the compiled version of the wp-rocket so that we can test functionality in mulitple environments and mulitple servers.

Daniyal-DigitalOcean avatar Dec 10 '25 08:12 Daniyal-DigitalOcean

@Daniyal-DigitalOcean Here you go wp-rocket.zip

jeawhanlee avatar Dec 10 '25 08:12 jeawhanlee

@jeawhanlee The plugin is requesting the license key on our testing the servers.

Daniyal-DigitalOcean avatar Dec 10 '25 09:12 Daniyal-DigitalOcean

We have tested the build in one of the clients server with valid license key and varnishlogs are showing 2 requests

*   << Request  >> 98356     
-   Begin          req 98355 rxreq
-   Timestamp      Start: 1765359164.890628 0.000000 0.000000
-   Timestamp      Req: 1765359164.890628 0.000000 0.000000
-   ReqStart       127.0.0.1 16496 a0
-   ReqMethod      PURGE
-   ReqURL         /.*
-   ReqProtocol    HTTP/1.1
-   ReqHeader      Host: wordpress-1559248-6047490.cloudwaysapps.com
-   ReqHeader      User-Agent: WordPress/6.9; https://wordpress-1559248-6047490.cloudwaysapps.com
-   ReqHeader      Accept: */*
-   ReqHeader      Accept-Encoding: deflate, gzip, br, zstd
-   ReqHeader      X-Purge-Method: regex
-   ReqHeader      Connection: close
-   ReqHeader      X-Forwarded-For: 127.0.0.1
-   VCL_call       RECV
-   ReqUnset       X-Forwarded-For: 127.0.0.1
-   ReqHeader      x-forwarded-for: 
-   VCL_return     synth
-   ReqUnset       Accept-Encoding: deflate, gzip, br, zstd
-   ReqHeader      Accept-Encoding: gzip
-   VCL_call       HASH
-   VCL_return     lookup
-   RespProtocol   HTTP/1.1
-   RespStatus     405
-   RespReason     Method Not Allowed
-   RespReason     This IP is not allowed to send PURGE requests.
-   RespHeader     Date: Wed, 10 Dec 2025 09:32:44 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 98356
-   VCL_call       SYNTH
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   VCL_return     deliver
-   Timestamp      Process: 1765359164.890689 0.000062 0.000062
-   RespHeader     Content-Length: 0
-   Storage        malloc Transient
-   RespHeader     Connection: close
-   Timestamp      Resp: 1765359164.890816 0.000189 0.000127
-   ReqAcct        250 0 250 229 0 229
-   End            

*   << Session  >> 98355     
-   Begin          sess 0 HTTP/1
-   SessOpen       127.0.0.1 16496 a0 127.0.0.1 8080 1765359164.890581 25
-   Link           req 98356 rxreq
-   SessClose      REQ_CLOSE 0.000
-   End            

*   << Request  >> 229470    
-   Begin          req 229469 rxreq
-   Timestamp      Start: 1765359164.894260 0.000000 0.000000
-   Timestamp      Req: 1765359164.894260 0.000000 0.000000
-   ReqStart       127.0.0.1 16502 a0
-   ReqMethod      PURGE
-   ReqURL         /.*
-   ReqProtocol    HTTP/1.0
-   ReqHeader      X-Real-IP: 127.0.0.1
-   ReqHeader      X-Forwarded-For: 127.0.0.1
-   ReqHeader      X-Client-Real-IP: 127.0.0.1
-   ReqHeader      Host: wordpress-1559248-6047490.cloudwaysapps.com
-   ReqHeader      X-Forwarded-Proto: http
-   ReqHeader      X-Forwarded-Host: wordpress-1559248-6047490.cloudwaysapps.com
-   ReqHeader      X-Application: wordpress
-   ReqHeader      X-App-User: wkksgmsrnr
-   ReqHeader      X-Version: latest
-   ReqHeader      Connection: close
-   ReqHeader      User-Agent: WordPress/6.9; https://wordpress-1559248-6047490.cloudwaysapps.com
-   ReqHeader      Accept: */*
-   ReqHeader      Accept-Encoding: deflate, gzip, br, zstd
-   ReqHeader      X-Purge-Method: regex
-   ReqUnset       X-Forwarded-For: 127.0.0.1
-   ReqHeader      X-Forwarded-For: 127.0.0.1, 127.0.0.1
-   VCL_call       RECV
-   ReqUnset       X-Forwarded-For: 127.0.0.1, 127.0.0.1
-   ReqHeader      x-forwarded-for: 127.0.0.1
-   ReqUnset       Host: wordpress-1559248-6047490.cloudwaysapps.com
-   ReqHeader      Host: wordpress-1559248-6047490.cloudwaysapps.com
-   VCL_return     purge
-   ReqUnset       Accept-Encoding: deflate, gzip, br, zstd
-   ReqHeader      Accept-Encoding: gzip
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       PURGE
-   VCL_return     synth
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespReason     Purged
-   RespHeader     Date: Wed, 10 Dec 2025 09:32:44 GMT
-   RespHeader     Server: Varnish
-   RespHeader     X-Varnish: 229470
-   VCL_call       SYNTH
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   VCL_return     deliver
-   Timestamp      Process: 1765359164.894396 0.000136 0.000136
-   RespHeader     Content-Length: 0
-   Storage        malloc Transient
-   RespHeader     Accept-Ranges: bytes
-   RespHeader     Connection: close
-   Timestamp      Resp: 1765359164.894488 0.000228 0.000092
-   ReqAcct        486 0 486 212 0 212
-   End            

One call is showing 405 and second one is showing 200.

Daniyal-DigitalOcean avatar Dec 10 '25 09:12 Daniyal-DigitalOcean

@Daniyal-DigitalOcean How did you test this? was it like below?

  • Fresh install & activate plugin
  • Clear and preload cache from WP Rocket dropdown in navbar?

jeawhanlee avatar Dec 10 '25 09:12 jeawhanlee

I have uploaded the .zip files from the wp-admin and replaced the current with the new version and then purge the cache from the wp-rocket by Clear and preload cache from WP Rocket dropdown in navbar

Daniyal-DigitalOcean avatar Dec 10 '25 09:12 Daniyal-DigitalOcean

@jeawhanlee Could you please confirm how will you plan to roll this update to the current clients that are using wp-rocket?

Daniyal-DigitalOcean avatar Dec 10 '25 09:12 Daniyal-DigitalOcean

@Daniyal-DigitalOcean It would be like every regular update if all things goes well, should be included in 3.20.3.

jeawhanlee avatar Dec 10 '25 10:12 jeawhanlee

@Daniyal-DigitalOcean Thank you for your support so far. We would like to know if the first request with the 405 is from WP Rocket or it's specific to your testing environment, to do this could you follow these steps to help us return the logs of ips being sent by WP Rocket.

  • Go to wp-rocket/inc/Addon/Varnish/Varnish.php
  • After this line: https://github.com/wp-media/wp-rocket/blob/bff588b3d101a1342bd2dcd8554c488e2dd23f1b/inc/Addon/Varnish/Varnish.php#L110
  • Add this new line: error_log('WPR Varnish Ips: ' . print_r($varnish_ip, true));
  • Enable debugging on WP if not enabled
    define( 'WP_DEBUG', true);
    define( 'WP_DEBUG_LOG', true );
    
  • Clear debug.log just to have a fresh log.
  • Clear and preload cache from WP Rocket dropdown in navbar?
  • Share the content of the log

It should look like this:

Image

jeawhanlee avatar Dec 10 '25 13:12 jeawhanlee

@jeawhanlee it is not possible for me to make these changes on the client server, Is it possible for you to authorise wp-rocket license on my email. https://eu.onetimesecret.com/secret/4ikx1uv5oe3kpzxjpk6br3iep50w3zo The link will be expired after one time access.

Daniyal-DigitalOcean avatar Dec 11 '25 09:12 Daniyal-DigitalOcean

@Daniyal-DigitalOcean You should get details for a single license in your mail.

jeawhanlee avatar Dec 12 '25 07:12 jeawhanlee

@Daniyal-DigitalOcean Just checking if you got the WPR details in your mail?

jeawhanlee avatar Dec 16 '25 09:12 jeawhanlee

Hi @jeawhanlee Thank you for the update. Could you please pause the work on the server, We are checking few changes again with our VCL which will resolve the issue with the purging on port 8080 too.

Additionally, I have got the access too.

Daniyal-DigitalOcean avatar Dec 16 '25 11:12 Daniyal-DigitalOcean

Blocking this till we get feedback from the cloudways team.

jeawhanlee avatar Dec 16 '25 11:12 jeawhanlee