ModSecurity icon indicating copy to clipboard operation
ModSecurity copied to clipboard

ngin reload memory leak

Open Hello-Linux opened this issue 4 years ago • 16 comments

It often leads memory leak on nginx reload using modsecurity branch 3.1-experimental and master,I hope that the official will fix this problem as soon as possible, which has a serious impact on the production environment. I have raised this serious problem several times. When will the official fix it?

I feel that this problem should be solved first, but why haven’t you fixed it after so long?

And There is also why this problem #2381 was closed without being resolved!!!

I suggest that you officially do a test yourself. First, create a lot of virtual hosts, each virtual host quotes the modsecurity rules separately, and then constantly reload, the memory leak should be reproduced!

help!help!help!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Hello-Linux avatar Apr 15 '21 06:04 Hello-Linux

I have the same memory leak problem with v3/master. I have also tried with v3/dev/3.1-experimental but there is a problem with some rules.

"modsecurity_rules_file" directive Rules error. File: /usr/local/coreruleset/rules/REQUEST-901-INITIALIZATION.conf. Line: 320. Column: 29. Action: ctl:forceRequestBodyVariable is not yet supported. in

ffais avatar May 08 '21 11:05 ffais

@ffais memory leak is just happening when the parser brakes?

zimmerle avatar May 17 '21 18:05 zimmerle

No this also happens also when the parser work properly.

ffais avatar May 18 '21 07:05 ffais

No this also happens also when the parser work properly.

I can confirm the issue on 3.1-experimental. However, that class of problem only happens in the warning framework, only available on 3.1-experimental. On v3/master warnings are treated as errors, the parser finishes abruptly, leaving no chance for leaks -- at least not on reload.

zimmerle avatar May 28 '21 01:05 zimmerle

We use Nginx with mod security in an autoscaling environment with nginx as a load balancer. We build it from open source. We recently upgraded to Nginx 1.18 w/ Modsecurity 3.0.4 and the memory leak is pretty severe. We have kept more instances running last 2 days to ensure there are not so frequent.

image

pcs9124 avatar Jun 06 '21 01:06 pcs9124

The same problem in Apache with ModSecurity v3.0.4 too, test show below

mod3_1000rps

willyamcts avatar Jun 22 '21 20:06 willyamcts

We use Nginx with mod security in an autoscaling environment with nginx as a load balancer. We build it from open source. We recently upgraded to Nginx 1.18 w/ Modsecurity 3.0.4 and the memory leak is pretty severe. We have kept more instances running last 2 days to ensure there are not so frequent.

image

Is this issue still happening with v3/master? What is your distro?

zimmerle avatar Jun 30 '21 16:06 zimmerle

The same problem in Apache with ModSecurity v3.0.4 too, test show below

mod3_1000rps

@willyamcts Apache conector for 3.x is not yet stable. Please use version 2.x with Apache.

zimmerle avatar Jun 30 '21 16:06 zimmerle

We use Nginx with mod security in an autoscaling environment with nginx as a load balancer. We build it from open source. We recently upgraded to Nginx 1.18 w/ Modsecurity 3.0.4 and the memory leak is pretty severe. We have kept more instances running last 2 days to ensure there are not so frequent. image

Is this issue still happening with v3/master? What is your distro?

We have only tried on v3.0.4, not master. nginx 1.18 on aws ami 2

We will build with master and report back.

pcs9124 avatar Jul 01 '21 07:07 pcs9124

We have only tried on v3.0.4, not master. nginx 1.18 on aws ami 2

We will build with master and report back.

thank you!

zimmerle avatar Jul 01 '21 11:07 zimmerle

I made some test building from master, memory leak is still present.

More details: nginx: 1.21.1 ModSecurity Commit SHA: faad65d3859c580a1aa428eb3009338452315f71 coreruleset: 3.3.2 Sites type: wordpress Sites number: ~20 ModSecurity enabled at server level.

ffais avatar Jul 07 '21 15:07 ffais

We have the same problem, I've tested it in v3/master and v3/dev/3.1 and I see the leak in both, I don´t see it in v3/dev/3.1-experimental nor in PR #2580. The more rules you load, the bigger the leak is. I've tested it in nginx 1.18.0 on Ubuntu 20.04 LTS.

kudrom avatar Jul 07 '21 17:07 kudrom

I checked both v3/dev/3.1-experimental and v3/master, it seems we still have the same problem. nginx/1.21.4 Ubuntu 18.04 Nginx up time : 1 day 18h Memory usage 3.5 GB 343 virtual hosts

aaishere avatar Nov 24 '21 20:11 aaishere

Hello there,

Sorry for the dig up, but I still observe the memory leak when I try to use NGINX / ModSecurity / OWASP CRS.

Here my environment: ModSecurity v3 Nginx Connector 1.0.2 ModSecurity v3.0.6 OWASP CRS 3.3.2 Ubuntu 18.04.4 LTS nginx 1.14.0

I know my environment is not all uptodate, but many people try on many different environments and all of them observe the memory leak.

I'm here to ask if there is a solution, because this issue run for a long time now, more than 1 year ?! I need it to be compatible with the OWASP CRS, and as mentioned by kudrom: https://github.com/SpiderLabs/ModSecurity/issues/2381#issuecomment-863227548 OWASP CRS and their rules depend on ctl:forceRequestBodyVariable which is not supported in the experimental branch (Thanks for the test by the way kudrom, it saves me some time)

Does anyone have some news ?

Best regards

labanana34 avatar Feb 24 '22 17:02 labanana34

Hello @labanana34,

Regarding ctl:forceRequestBodyVariable: it isn't really relevant to the topic in this issue. It's helpful to other users searching about things if comments are directly related to the issues in which they are posted. Note, however, that there is an open issue related to you inquiry here: https://github.com/SpiderLabs/ModSecurity/issues/2146 .

Regarding memory leaks on rule reload: other things have taken priority recently, but I hope to spend some time on that in the post-v3.0.7 period.

martinhsv avatar Feb 28 '22 16:02 martinhsv

@labanana34 could you try https://github.com/SpiderLabs/ModSecurity-nginx/pull/277?

liudongmiao avatar Mar 22 '22 16:03 liudongmiao

Still happening on ModSecurity v3 Nginx Connector 1.0.3

proginter avatar Nov 08 '22 08:11 proginter

Closing as duplicate

martinhsv avatar Dec 21 '22 19:12 martinhsv