Fatal memory exhaustion error when updating firewall rules via REST API

[tschunk]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• [x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• [x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

On OPNsense 24.1.9 when using the REST API to manage firewall rules, updating a rule via REST API fails with a fatal error:

> POST /api/firewall/filter/set_rule/<uuid>

< HTTP/2 200
< Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 4096 bytes) in /usr/local/opnsense/mvc/app/models/OPNsense/Base/FieldTypes/BaseListField.php on line 156

This only occurs if a lot of rules are present and a rule is tried to be updated via REST API. In my tests the threshold was at about 300 rules. Creation and deletion of rules via API still works fine.

Notes:

• On 23.7 with os-firewall plugin the error does not occur.
• When testing with very simple rules (PASS any 1.1.1.1 -> 2.2.2.2 instead of PASS TCP 1.1.1.1 -> 2.2.2.2:80) the error occured at a much higher threshold.
• During my debugging I came across the fix for #6978 (commit e36123c) which may be a likely candidate for having introduced the issue.

To Reproduce

Steps to reproduce the behavior:

1. Use REST API to create 350 firewall rules (I tested with PASS TCP 1.1.1.1 -> 2.2.2.2:80)
2. Use REST API (or UI: Firewall -> Automation) to update one of the rules (e.g. change description)
3. API request fails with fatal error and rule is not changed.

Or use this script:

#!/bin/bash
set -ex

firewall="https://CHANGEME"
key=CHANGEME
secret=CHANGEME

# create 300 firewall rules
for i in $(seq 300); do
  res=$(curl -ks "${firewall}/api/firewall/filter/add_rule" \
    --user "${key}:${secret}" \
    -X POST -H 'Content-Type: application/json' \
    --data-raw '{"rule":{"enabled":"1","sequence":"1","action":"pass","quick":"1","interface":"lan","direction":"in","ipprotocol":"inet","protocol":"TCP","source_net":"1.1.1.1","source_port":"","source_not":"0","destination_net":"2.2.2.2","destination_not":"0","destination_port":"80","gateway":"","log":"0","categories":"","description":"create"}}'
  )
done

# uuid of last created rule
uuid="$(echo "$res" | sed -re 's/.*"uuid":"(.+)".*/\1/')"

# try to update a rule
# will print `Fatal error: Allowed memory size of 1073741824 bytes exhausted`
curl -ksv "${firewall}/api/firewall/filter/set_rule/${uuid}" \
  --user "${key}:${secret}" \
  -X POST -H 'Content-Type: application/json' \
  --data-raw '{"rule":{"enabled":"1","sequence":"1","action":"pass","quick":"1","interface":"lan","direction":"in","ipprotocol":"inet","protocol":"TCP","source_net":"1.1.1.1","source_port":"","source_not":"0","destination_net":"2.2.2.2","destination_not":"0","destination_port":"80","gateway":"","log":"0","categories":"","description":"update"}}'

Expected behavior

Using REST API to update a firewall rule does not fail with an error and rule is correctly updated.

Describe alternatives you considered

Using the API to delete the rule and create a new rule with the required changes does still work, but this is very cumbersome.

Relevant log files

Only the above error message is returned in HTTP API call and also logged to /tmp/PHP_errors.log. I could not find any other relevant logs. I also submitted this error via System: Firmware: Reporter

Environment

OPNsense 24.1.9_4 (amd64) Tested in virtual machine (qemu)

[AdSchellevis]

the set and add actions are roughly the same, so the question is if you can also trigger your issue when trying to set a non-existing uuid (just remove the one created in step one and update the same uuid).

Since set is an upsert operation, when the entry doesn't exist, it should do roughly same as add in this case:

https://github.com/opnsense/core/blob/c1f51000f997ad252d9176008c33ba5ca640d511/src/opnsense/mvc/app/controllers/OPNsense/Base/ApiMutableModelControllerBase.php#L537-L544

[tschunk]

Hi!

yes, I get the same error when using set with a unknown UUID.

script to reproduce:

#!/bin/bash
firewall="https://CHANGEME"
key=CHANGEME
secret=CHANGEME

set -ex
uuid="$(python3 -c 'import uuid; print(uuid.uuid4());')"

# try to upsert a rule
# will print `Fatal error: Allowed memory size of 1073741824 bytes exhausted`
curl -ksv "${firewall}/api/firewall/filter/set_rule/${uuid}" \
  --user "${key}:${secret}" \
  -X POST -H 'Content-Type: application/json' \
  --data-raw '{"rule":{"enabled":"1","sequence":"1","action":"pass","quick":"1","interface":"lan","direction":"in","ipprotocol":"inet","protocol":"TCP","source_net":"1.1.1.1","source_port":"","source_not":"0","destination_net":"2.2.2.2","destination_not":"0","destination_port":"80","gateway":"","log":"0","categories":"","description":"update"}}'

Output:

++ python3 -c 'import uuid; print(uuid.uuid4());'
+ uuid=c0c74609-a349-4088-9f0c-fcf7145ffc2d
+ curl -ksv https://XXXXXXX/api/firewall/filter/set_rule/c0c74609-a349-4088-9f0c-fcf7145ffc2d --user XXXXXXX -X POST -H 'Content-Type: application/json' --data-raw '{"rule":{"enabled":"1","sequence":"1","action":"pass","quick":"1","interface":"lan","direction":"in","ipprotocol":"inet","protocol":"TCP","source_net":"1.1.1.1","source_port":"","source_not":"0","destination_net":"2.2.2.2","destination_not":"0","destination_port":"80","gateway":"","log":"0","categories":"","description":"update"}}'
[...]
> POST /api/firewall/filter/set_rule/c0c74609-a349-4088-9f0c-fcf7145ffc2d HTTP/2
> Host: XXXXXXX
> Authorization: Basic XXXXXXXXX
> User-Agent: curl/8.6.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 332
>
< HTTP/2 200
< content-type: text/html; charset=UTF-8
< content-length: 191
< date: Mon, 15 Jul 2024 10:16:00 GMT
< server: OPNsense
<

Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 2097160 bytes) in /usr/local/opnsense/mvc/app/models/OPNsense/Base/FieldTypes/BaseListField.php on line 156

PS: I tested this now with latest version 24.1.10_2

[alexatothermo]

Just want to add to this issue that I'm seeing this when working the wireguard plugin via the http API.

When calling POST /wireguard/client/addClient I receive Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 4096 bytes) in /usr/local/opnsense/mvc/app/models/OPNsense/Base/FieldTypes/BaseListField.php on line 156

I too have hundreds of wireguard clients managed in opnsense.

EDIT: I worked around this temporarily (it will be gone after the next update and has to be re-applied then) by modifying memory_limit to 2G in /usr/local/opnsense/service/templates/OPNsense/WebGui/php.ini.

Still it seems very wrong that with around 300 wireguard clients such an operation would exceed the memory limit of 1G.

[AdSchellevis]

(very) large config.xml?

[n0nalc0h0lic]

Hi, I have the same issue with the Version 24.7.6 (amd64) on qemu vm (2 cores, 4GB).

Workaround in my case was to increase the memory_limit, as mentioned before, and to increase the vms memory to 8GB.

[AdSchellevis]

could be same as https://github.com/opnsense/core/issues/8123

[OPNsense-bot]

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.