ModSecurity Variables Automatically Expire After 1 Minute Without ExpireVar Setting
Describe the bug
Variables created using setvar in ModSecurity rules are being automatically deleted after 1 minute, even without explicitly setting expirevar. This occurs when trying to implement custom rules for login attempt tracking without using CRS.
Steps to reproduce
- Use the official OWASP ModSecurity CRS Docker image (owasp/modsecurity-crs:4.7.0-nginx-202410090410)
- Override
/etc/nginx/templates/modsecurity.d/setup.conf.templatewith custom configuration:- Comment out CRS includes:
# Include /etc/modsecurity.d/owasp-crs/crs-setup.conf # Include /etc/modsecurity.d/owasp-crs/rules/*.conf - Add custom rule file:
Include /opt/on_pre_llm/extra_lockout_rule.conf
- Comment out CRS includes:
- Implement custom rule for tracking login attempts:
SecRule REQUEST_URI "@beginsWith /console/api/login" \ "id:100098,phase:1,pass,\ chain" SecRule &IP.failed_attempts "@eq 0" \ "setvar:IP.failed_attempts=0,\ setvar:IP.is_locked=0,\ msg:'Initialization - Failed attempts: %{IP.failed_attempts}'" - Wait for 1 minute
- Make another request to the login endpoint
Expected behaviour
- The
IP.failed_attemptsandIP.is_lockedvariables should persist indefinitely until explicitly cleared - The initialization message should only appear on the first request when the variables don't exist
Actual behaviour
- Variables are automatically deleted after 1 minute
- The initialization message appears again after 1 minute, indicating that the variables have been reset
- This behavior occurs with both IP and GLOBAL collection variables
- No
expirevaris explicitly set in the rules
Additional context
- This issue prevents implementing proper login lockout functionality as the attempt counter keeps resetting
- Confirmed that custom rules are working as the messages appear in the Docker logs
- CORS is not active and not related to this issue
Your Environment
- CRS version: Not using CRS (disabled)
- Paranoia level setting: N/A (custom rules only)
- ModSecurity version: As included in owasp/modsecurity-crs:4.7.0-nginx-202410090410
- Web Server and version: nginx (version from Docker image)
- Operating System and version: Ubuntu 24.04
- Docker image: owasp/modsecurity-crs:4.7.0-nginx-202410090410
@theseion Do you have a clue on why this might be happening?
@takumi-ricoh Are you sure you are using the right syntax? 🤔 What I'm seeing in CRS is that we use :. E.g.
SecRule &IP:failed_attempts "@eq 0" \
"setvar:'IP.failed_attempts=0',\
setvar:'IP.is_locked=0',\
msg:'Initialization - Failed attempts: %{IP.failed_attempts}'"
@fzipi Thank you for the advice. I tried it but it still didn't work.
I believe I'm in a situation where the application I want to use has no login lockout feature, and I have to implement it through WAF, so I'd like to solve this using ModSecurity somehow.
※ Given that there's documentation like this, implementing these (brute-force) measures using ModSecurity isn't strange, right? https://docs.mirantis.com/mcp/q4-18/mcp-security-best-practices/use-cases/brute-force-prevention/create-brute-force-rules.html
Current Code
When using the following code, if accessed within 1 minute, the count increases and in the logs, "Current number of failures" incrementally goes up like 1⇒2⇒..., but after 1 minute passes, it resets back to 1.
print IP.failed_attempts as Current number of failures:
SecRule REQUEST_URI "@beginsWith /console/api/login" \
"id:100098,phase:1,pass,\
chain"
SecRule &IP:failed_attempts "@eq 0" \
"setvar:'IP.failed_attempts=0',\
setvar:'IP.is_locked=0',\
msg:'Initialization - Failed attempts: %{IP.failed_attempts}'"
SecRule REQUEST_URI "@beginsWith /console/api/login" \
"id:100102,phase:5,pass,\
chain"
SecRule RESPONSE_STATUS "@rx ^4\d{2}$" \
"setvar:'IP.failed_attempts=+1',\
log,\
msg:'Login failure count - Current number of failures: %{IP.failed_attempts}'"
docker log
※ ■■■■■■■■■■■■ hoge ■■■■■■■■■■■■ is my extra comment
- 1st access
- 100098 & 100102
-
Current number of failures: 1
- 2nd access (immediately after)
- 100102
-
Current number of failures: 2
- 3rd access (more than 1 minute after 2nd access)
- 100098 & 100102
-
Current number of failures: 1
- 4th access (more than 1 minute after 3rd access)
- 100098 & 100102
-
Current number of failures: 1
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/90-copy-modsecurity-config.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/91-update-resolver.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/92-update-real_ip.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/93-update-proxy-ssl-config.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/94-activate-plugins.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/95-activate-rules.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/11/01 01:49:58 [warn] 1#1: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/conf/server.crt"
nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/conf/server.crt"
2024/11/01 01:49:58 [notice] 1#1: ModSecurity-nginx v1.0.3 (rules loaded inline/local/remote: 0/12/0)
2024/11/01 01:49:58 [notice] 1#1: libmodsecurity3 version 3.0.13
■■■■■■■■■■■■ This is 1st access ⇒ Current number of failures: 1 ■■■■■■■■■■■■
10.228.107.168 - - [01/Nov/2024:01:50:02 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov 1 01:50:02 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":59596,"host_ip":"172.19.0.10","host_port":443,"unique_id":"173042580233.389326","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:50:02 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Initialization - Failed attempts: 0","details":{"match":"Matched \"Operator `Eq' with parameter `0' against variable `IP:failed_attempts' (Value: `0' )","reference":"o0,18v5,18","ruleId":"100098","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"7","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}},{"message":"Login failure count - Current number of failures: 1","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}
■■■■■■■■■■■■ 2nd access (immediately after) ⇒ Current number of failures: 2 ■■■■■■■■■■■■
10.228.107.168 - - [01/Nov/2024:01:50:04 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov 1 01:50:04 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":59596,"host_ip":"172.19.0.10","host_port":443,"unique_id":"17304258047.838869","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:50:04 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Login failure count - Current number of failures: 2","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}
■■■■■■■■■■■■ 3rd access (more than 1 minute after 2nd access) ⇒ Current number of failures: 1 ■■■■■■■■■■■■
10.228.107.168 - - [01/Nov/2024:01:51:14 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov 1 01:51:14 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":52208,"host_ip":"172.19.0.10","host_port":443,"unique_id":"173042587458.408275","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:51:14 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Initialization - Failed attempts: 0","details":{"match":"Matched \"Operator `Eq' with parameter `0' against variable `IP:failed_attempts' (Value: `0' )","reference":"o0,18v5,18","ruleId":"100098","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"7","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}},{"message":"Login failure count - Current number of failures: 1","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}
■■■■■■■■■■■■ 4th access (more than 1 minute after 3rd access) ⇒ Current number of failures: 1 ■■■■■■■■■■■■
10.228.107.168 - - [01/Nov/2024:01:52:36 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov 1 01:52:36 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":38452,"host_ip":"172.19.0.10","host_port":443,"unique_id":"173042595696.346467","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:52:36 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Initialization - Failed attempts: 0","details":{"match":"Matched \"Operator `Eq' with parameter `0' against variable `IP:failed_attempts' (Value: `0' )","reference":"o0,18v5,18","ruleId":"100098","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"7","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}},{"message":"Login failure count - Current number of failures: 1","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}
Strange. You didn't post where your use initcol to initialize the IP collection. Could you post that please?
@theseion
Thank you for your response.
I'm not using initcol, and there are only the two rules I wrote above.
I'm not really familiar with modsecurity, so I feel a bit bad asking, but... do I need to use initcol?😓
I have created minimal code files for reproduction verification. As I'm not familiar with ModSecurity syntax, there might be errors in the code itself.
Usage Instructions:
- Unzip the attached zip file
- Run docker compose up -d
- A container named modsecurity_test will start
- Access port 80 through a browser using localhost or the host's IP address
- An nginx 404 page will be displayed as shown below
Verification Steps:
- Display the docker logs for modsecurity_test
- Reload the 404 page above and check the following section: message":"********** Count Up ********* - Current: 1"
- If you reload the 404 page without waiting, the Current value will increment like 1⇒2⇒3...
- If you wait for 1 minute before reloading, the Current value will reset to 1.
Additional Note:
Originally, CRS rules were commented out in my setup.conf, but this time I left them unchanged to minimize modifications to the original settings. (Please refer to the attached files)
initcol must be used to initialize the collections. CRS will do this for you here: https://github.com/coreruleset/coreruleset/blob/d33dad89052ecc531641b047efcaae339c254362/rules/REQUEST-901-INITIALIZATION.conf#L319.
- You mustn't use
initcolmore than once - Use the
initcolrule fromREQUEST-901-INITIALIZATION.conf - Try playing with
SecCollectionTimeout. The default should be 1 hour.
If you still can't get it working, I'll be happy to take a look.
Thank you for your reply. I tried making the following modifications as suggested:
- Removed the SecAction using initcol in countup.conf
- Set the environment variable MANUAL_MODE to "true"
- Created crs-setup.override.conf and uncommented line 430 to enable enable_default_collections in Initialize Default Collections section of crs-setup.conf
After implementing these settings, for some reason the SecAction (id:900130) on line 430 of crs-setup.override.conf was not executed. When I checked the files in the running container, crs-setup.conf was correctly overwritten by crs-setup.override.conf. There might be something wrong with how I configured these settings. countup_test_crs_docker_2.zip
After trying all of this without resolving the issue, I discovered that I could achieve what I wanted using Nginx's limit_req functionality, so I decided to go with that approach instead. https://blog.nginx.org/blog/rate-limiting-nginx
Thank you for your help. I'll close this issue.