Huge number of SCAN commands sent to REDIS

[germanftorres]

Hi,

I'm using caddy + souin plugin + go-redis storage as reverse proxy to an imgproxy service.

extract from Dockerfile

FROM caddy:2.10.2-builder-alpine

xcaddy build \
    --with github.com/darkweak/souin/plugins/caddy \
    --with github.com/darkweak/storages/go-redis/caddy

Caddyfile

{
	auto_https off

	cache {
		redis {
			configuration {
				Addrs {$REDIS_ADDRR}
				Username {$REDIS_USERNAME}
				Password {$REDIS_PASSWORD}
				PoolSize 50
				ClientName souin-redis
			}
		}
	}
}

:8081 {
	cache {

		ttl 86400s

		allowed_http_verbs GET

		mode bypass_request
	}
	reverse_proxy :8080
}

I'm using Fly.io's managed redis service (Upstash redis) and in the service console I can see a huge amount of SCAN commands that are being sent to the instance from the region where my app is hosted.

I was wondering if somehow the souin plugin is polling all the keys (via MapKeys or ListKeys) continuosly and if it's reasonably that the caching solution is issuing so many SCAN commands in comparison with the other commands (GET, SET) that seem to be ok with the little traffic I'm sending to a test server.

Thanks

Germán

PS: There's an issue documented at Flyio.s troubleshooting docs about Unexpected Increase in Command Count to REDIS from their own console display and monitoring purposes, but doesn't seem related as the increase is very modest in comparison to this issue.

[darkweak]

Hey @germanftorres I'm working on a way to reduce the number of mapping eviction and use another thing than the SCAN command.

[cdaguerre]

hi @darkweak. we are seeing the same problem in production. It appears this wasnt the case until recently. Can you tell us which commit introduced this? We would like to roll back ;)

EDIT: I think we found it: https://github.com/darkweak/souin/commit/bd94dfc0ce71ab063281aac4fa999faa57c72410

[darkweak]

I think that's the registering mapping eviction and run every minutes to purge IDX_ keys.

[cdaguerre]

Hey @darkweak, is there anything we can do to help?