IO error when running with a key wildcard

[waingro]

Version: Nginx Cache Purge 0.1.10 OS: Ubuntu 20.04.6 LTS

Example command: /usr/local/bin/nginx-cache-purge /var/cache/nginx 1:2 'https*' Relevant Config:

proxy_cache_path /var/cache/nginx keys_zone=disk_cache:100m levels=1:2 inactive=24h max_size=1G;
proxy_cache_key "$scheme$host$request_uri";

Expectation. Running the script once will clear all cache with a wildcard option.

Actual. The script usually returns: "Error: Kind(InvalidData)" and requires running a few more times before it successfully runs without an error. When it runs with no error it works as expected. It appears to happen more when I run with a wildcard 'httpsEXAMPLE.HOSTNAME*' instead of just '*'.

Troubleshooting steps attempted:

1. Script is running as the www-data user. /var/cache/nginx is owned by this user and is the same group.
2. Tried running the command locally with sudo and same error. Permission errors return a different error.
3. Tried running with and without quotes around the wildcard. Seems to make no difference.

Have used the following content block workaround: `content_by_lua_block { local count = 0 local limit = 10 local exitStatus

      # Try to run this script up to ten times
      while not exitStatus and count < limit do
          exitStatus = os.execute("/usr/local/bin/nginx-cache-purge /var/cache/nginx 1:2 '"..ngx.var.my_cache_key.."'")
          count = count + 1
      end

      if exitStatus then
          ngx.status = ngx.HTTP_OK
      else
          # On failure return the number of times the script tried to execute
          ngx.say("Count: "..count.." Status: "..(exitStatus and "success" or "failure"))
          ngx.status ngx.HTTP_BAD_REQUEST
      end
      
      # Exit
      ngx.exit(ngx.status)
  }`

Let me know if I can provide any additional information to help troubleshoot this.

[khuezy]

I've noticed a similar issue. Another problem is if you're purging /path/to and there's data at: /path/to/a/b, it errors saying that the directory is not empty.

@magiclen Thanks for this great project, do you have any plans to continue development or is this project in "archive" mode?

[magiclen]

@waingro Thanks for your report. But I cannot reproduce your issue. A new version has been released. Would you like to try it out?

@khuezy Thanks for your report. I believe this issue has been resolved in the new version.

[khuezy]

Thanks for the fix @magiclen!

On a related note, can I add a feature request to allow for the negation "!" path? My use case is that I want to purge everything but the optimized image route, eg: !/_next/image*

[khuezy]

I'm having issues w/ the wildcard purge on latest (0.3.1)

Command: /usr/local/bin/nginx-cache-purge purge /data/nginx-cache 1:2 *

error: unexpected argument 'bin' found

I've tried wrapping it in quotes and it runs, but nothing is purged (status 44 is returned, indicating nothing was purged)

[magiclen]

I'm having issues w/ the wildcard purge on latest (0.3.1)

Command: /usr/local/bin/nginx-cache-purge purge /data/nginx-cache 1:2 *

error: unexpected argument 'bin' found

I've tried wrapping it in quotes and it runs, but nothing is purged (status 44 is returned, indicating nothing was purged)

So there are directories and files in /data/nginx-cache but they still exist?

[khuezy]

Yes, when I load the page, all the resources at HITs When I purge a specific path eg: /_next/*, then all the static contents under that path are purged and the first response are MISSes

[magiclen]

I cannot reproduce that. It is weird. When key is *, the remove_all_files_in_directory function is used. And it does something like rm -rf cache_path/*. All files in cache_path should be directly removed.

[magiclen]

@khuezy A new version has been released, introducing the --exclude-keys option, which can be set to prevent caches with a specific key pattern from being purged.

On a related note, can I add a feature request to allow for the negation "!" path? My use case is that I want to purge everything but the optimized image route, eg: !/_next/image*

In this case, the purge command should be something like nginx-cache-purge p /data/nginx-cache 1:2 '*' -e '/_next/image*'.

[khuezy]

Awesome thanks so much! I'll test that out this weekend. As for the error I had yesterday, it was user error... my script was: nginx-cache-purge p /data/nginx-cache 1:2 "*" The double quotes around the * messed it up.

[khuezy]

@magiclen nginx-cache-purge p /data/nginx-cache 1:2 '*' -e '/_next/image*' this mostly works but only a few of the images are purged, the rest remains and are HITs

[magiclen]

@khuezy It should remove all caches except for those whose key starts with /_next/image.

[waingro]

@magiclen Thank you for releasing an update, I will try it out in the next two weeks and report back to you on the results.

[gokhansengun]

Hey @magiclen, if you are interested, I have curated a minimal reproduction case for this issue. You can find it below.

https://github.com/gokhansengun/nginx-cache-purge-repro