caddyserver
What is `SURROGATE_`?
It is recording 50MB large value to Redis and that's taking a significant toll on the CPU.
This is proving to be a big problem, and I am surprised I cannot find more information about it.
Because this key grows to such a crazy large size, and because it is using Redis as the backend, every time it is set/retrieved, it is causing slowdown of every other session key retrieval.
Can you share the names of the keys in your Redis instance? And perhaps also the commit hash of cache-handler and github.com/darkweak/storages, to see if the mentioned memory leak issue is at play or still relevant, even after the fix.
I just updated my build/config to the following:
FROM caddy:builder-alpine AS caddy-builder
- RUN xcaddy build --with github.com/caddyserver/cache-handler --with github.com/darkweak/storages/redis/caddy
+ RUN xcaddy build --with github.com/darkweak/souin/plugins/caddy --with github.com/darkweak/storages/redis/caddy
{
cache {
ttl 100s
stale 3h
allowed_http_verbs GET HEAD
redis {
configuration {
InitAddress glama-redis.internal:6379
Username default
ClientName caddy
ForceSingleClient true
AlwaysPipelining true
}
}
key {
disable_host
disable_method
+ hash
}
+ max_cacheable_body_bytes 1048576
}
}
also, I've noticed that some of the cache keys are extremely long, e.g., Those represent React Router manifest:
https://glama.ai/__manifest?p=%2F&p=%2Fblog&p=%2Fblog%2F2024-11-25-model-context-protocol-quickstart&p=%2Fcareers&p=%2Fchat&p=%2Fdiscord&p=%2Fgateway&p=%2Fgateway%2Fdocs&p=%2Fgateway%2Freference&p=%2Fmcp&p=%2Fmcp%2Fapi&p=%2Fmcp%2Fclients&p=%2Fmcp%2Fdiscord&p=%2Fmcp%2Fservers&p=%2Fmcp%2Fservers%2F%40302ai%2F302_browser_use_mcp&p=%2Fmcp%2Fservers%2F%40AdsPower%2Flocal-api-mcp-typescript&p=%2Fmcp%2Fservers%2F%40CoinStatsHQ%2Fcoinstats-mcp&p=%2Fmcp%2Fservers%2F%40Infisical%2Finfisical-mcp-server&p=%2Fmcp%2Fservers%2F%40MiniMax-AI%2FMiniMax-MCP-JS&p=%2Fmcp%2Fservers%2F%40OctagonAI%2Foctagon-mcp-server&p=%2Fmcp%2Fservers%2F%40OctoMind-dev%2Foctomind-mcp&p=%2Fmcp%2Fservers%2F%40Scrapezy%2Fmcp&p=%2Fmcp%2Fservers%2F%40TimLukaHorstmann%2Fmcp-weather%2Ftools%2Fweather-get_hourly&p=%2Fmcp%2Fservers%2F%40VapiAI%2Fmcp-server&p=%2Fmcp%2Fservers%2F%40agentset-ai%2Fmcp-server&p=%2Fmcp%2Fservers%2F%40apw124%2Flogseq-mcp%2Ftools%2Finsert_block&p=%2Fmcp%2Fservers%2F%40binalyze%2Fair-mcp&p=%2Fmcp%2Fservers%2F%40chargebee%2Fagentkit&p=%2Fmcp%2Fservers%2F%40cloudflare%2Fplaywright-mcp&p=%2Fmcp%2Fservers%2F%40databutton%2Fdatabutton-mcp&p=%2Fmcp%2Fservers%2F%40datastax%2Fastra-db-mcp&p=%2Fmcp%2Fservers%2F%40glifxyz%2Fglif-mcp-server&p=%2Fmcp%2Fservers%2F%40hackle-io%2Fhackle-mcp&p=%2Fmcp%2Fservers%2F%40heroku%2Fheroku-mcp-server&p=%2Fmcp%2Fservers%2F%40iaptic%2Fmcp-server-iaptic&p=%2Fmcp%2Fservers%2F%40intsig-textin%2Ftextin-mcp&p=%2Fmcp%2Fservers%2F%40kohasummons%2Fchaingpt-mcp%2Ftools%2Fchaingpt_get_ai_crypto_news&p=%2Fmcp%2Fservers%2F%40luminati-io%2Fbrightdata-mcp&p=%2Fmcp%2Fservers%2F%40makeplane%2Fplane-mcp-server&p=%2Fmcp%2Fservers%2F%40mettamatt%2Fcode-reasoning%2Ftools%2Fcode-reasoning&p=%2Fmcp%2Fservers%2F%40mobile-next%2Fmobile-mcp&p=%2Fmcp%2Fservers%2F%40modelcontextprotocol%2Fgithub&p=%2Fmcp%2Fservers%2F%40otc-ai%2Fmcp-otc&p=%2Fmcp%2Fservers%2F%40pinecone-io%2Fpinecone-mcp&p=%2Fmcp%2Fservers%2F%40pinkpixel-dev%2Ftaskflow-mcp%2Ftools%2Fplan_task&p=%2Fmcp%2Fservers%2F%40playcanvas%2Feditor-mcp-server&p=%2Fmcp%2Fservers%2F%40powerdrillai%2Fpowerdrill-mcp&p=%2Fmcp%2Fservers%2F%40sanity-io%2Fsanity-mcp-server&p=%2Fmcp%2Fservers%2F%40tavily-ai%2Ftavily-mcp&p=%2Fmcp%2Fservers%2F%40upstash%2Fmcp-server&p=%2Fmcp%2Fservers%2F%40vectorize-io%2Fvectorize-mcp-server&p=%2Fmcp%2Fservers%2F%40yuhuacheng%2Ftidal-mcp%2Ftools%2Fcreate_tidal_playlist&p=%2Fmcp%2Fservers%2Fcategories%2Fagent-orchestration&p=%2Fmcp%2Fservers%2Fcategories%2Fapp-automation&p=%2Fmcp%2Fservers%2Fcategories%2Fautonomous-agents&p=%2Fmcp%2Fservers%2Fcategories%2Fblockchain&p=%2Fmcp%2Fservers%2Fcategories%2Fbrowser-automation&p=%2Fmcp%2Fservers%2Fcategories%2Fcloud-platforms&p=%2Fmcp%2Fservers%2Fcategories%2Fcode-execution&p=%2Fmcp%2Fservers%2Fcategories%2Fcommunication&p=%2Fmcp%2Fservers%2Fcategories%2Fcontent-management-systems&p=%2Fmcp%2Fservers%2Fcategories%2Fcryptocurrency&p=%2Fmcp%2Fservers%2Fcategories%2Fcustomer-data-platforms&p=%2Fmcp%2Fservers%2Fcategories%2Fdata-platforms&p=%2Fmcp%2Fservers%2Fcategories%2Fdatabases&p=%2Fmcp%2Fservers%2Fcategories%2Fdeveloper-tools&p=%2Fmcp%2Fservers%2Fcategories%2Fdocumentation-access&p=%2Fmcp%2Fservers%2Fcategories%2Ffile-systems&p=%2Fmcp%2Fservers%2Fcategories%2Ffinance&p=%2Fmcp%2Fservers%2Fcategories%2Fhybrid&p=%2Fmcp%2Fservers%2Fcategories%2Fimage-and-video-processing&p=%2Fmcp%2Fservers%2Fcategories%2Flocal-only&p=%2Fmcp%2Fservers%2Fcategories%2Fmonitoring&p=%2Fmcp%2Fservers%2Fcategories%2Fmultimedia-processing&p=%2Fmcp%2Fservers%2Fcategories%2Fproject-management&p=%2Fmcp%2Fservers%2Fcategories%2Frag-systems&p=%2Fmcp%2Fservers%2Fcategories%2Fremote-capable&p=%2Fmcp%2Fservers%2Fcategories%2Fresearch-and-data&p=%2Fmcp%2Fservers%2Fcategories%2Fsearch&p=%2Fmcp%2Fservers%2Fcategories%2Fsecurity-and-iam&p=%2Fmcp%2Fservers%2Fcategories%2Ftesting-and-qa-tools&p=%2Fmcp%2Fservers%2Fcategories%2Ftext-to-speech&p=%2Fmcp%2Fservers%2Fcategories%2Fvector-databases&p=%2Fmcp%2Fservers%2Fcategories%2Fversion-control&p=%2Fmcp%2Fservers%2Fcategories%2Fweb-scraping&p=%2Fmcp%2Fservers%2Fcategories%2Fweb3&p=%2Fmcp%2Ftools&p=%2Fmodels&p=%2Fplayground&p=%2Fpolicies%2Fprivacy-policy&p=%2Fpolicies%2Fsecurity-policy&p=%2Fpolicies%2Fterms-of-service&p=%2Fpricing&p=%2Fsign-in&p=%2Fstatus&p=%2Fsupport&version=78ced558
I am hoping that hash option will help here.
Waiting for changes to deploy.
I've noticed that some of the cache keys are extremely long
this can be fixed with template by using various placeholders in caddy in here like:
key {
template "{http.request.uri.query.url}"
}
You can now disable the surrogate keys system using disable_surrogate_key on the latest version (v1.7.7) of Souin, soon available in the cache-handler repository.
