process_event_rules() broke all events when condition evaluation contains None values.

[ravenrs]

Deployment Type

Self-hosted

NetBox Version

v4.2.2

Python Version

3.12

Steps to Reproduce

1. Install Netbox 4.2.2 with python 3.12 using documentation. here is my system config: netbox.json
2. Create site in web interface (all next action I will do in web interface). Here is needed data:

{ "id": 2, "url": "http://127.0.0.1:8000/api/dcim/sites/2/", "display_url": "http://127.0.0.1:8000/dcim/sites/2/", "display": "test_issue", "name": "test_issue", "slug": "test_issue", "status": { "value": "active", "label": "Active" }, "region": null, "group": null, "tenant": null, "facility": "", "time_zone": null, "description": "", "physical_address": "", "shipping_address": "", "latitude": null, "longitude": null, "comments": "", "asns": [], "tags": [], "custom_fields": {}, "created": "2025-06-04T08:13:03.378951Z", "last_updated": "2025-06-04T08:13:03.378995Z", "circuit_count": 0, "device_count": 0, "prefix_count": 0, "rack_count": 0, "virtualmachine_count": 0, "vlan_count": 0 }

3. Create webhook:

{ "id": 3, "url": "http://127.0.0.1:8000/api/extras/webhooks/3/", "display_url": "http://127.0.0.1:8000/extras/webhooks/3/", "display": "test_issue_wb", "name": "test_issue_wb", "description": "", "payload_url": "http://localhost:9000", "http_method": "POST", "http_content_type": "application/json", "additional_headers": "", "body_template": "", "secret": "", "ssl_verification": true, "ca_file_path": null, "custom_fields": {}, "tags": [], "created": "2025-06-04T08:16:29.243488Z", "last_updated": "2025-06-04T08:16:29.243519Z" }

4. Create event:

{ "id": 4, "url": "http://127.0.0.1:8000/api/extras/event-rules/4/", "display_url": "http://127.0.0.1:8000/extras/event-rules/4/", "display": "test_issue_event_without_condition.", "object_types": [ "dcim.site" ], "name": "test_issue_event_without_condition.", "enabled": true, "event_types": [ "object_updated" ], "conditions": null, "action_type": { "value": "webhook", "label": "Webhook" }, "action_object_type": "extras.webhook", "action_object_id": 3, "action_object": { "id": 3, "url": "http://127.0.0.1:8000/api/extras/webhooks/3/", "display": "test_issue_wb", "name": "test_issue_wb", "description": "" }, "description": "", "custom_fields": {}, "tags": [], "created": "2025-06-04T08:17:07.624292Z", "last_updated": "2025-06-04T08:17:25.888092Z" }

5. Go to terminal, go to the netbox directory, activate virtual env, run manage.py command for webhook_receiver.

password@Air cd ~Documents/netbox-4.2.2/netbox password@Air ~/Documents/netbox-4.2.2 source venv/bin/activate
((venv) ) password@Airr ~/Documents/netbox-4.2.2  python netbox/manage.py webhook_receiver Listening on port http://localhost:9000. Stop with CONTROL-C.

6. Edit created site, add comment - “test”
7. Check web hook receiver and make sure that you receive changes

Wed, 04 Jun 2025 08:20:59 GMT 127.0.0.1 "POST / HTTP/1.1" 200 - Host: localhost:9000 Accept-Encoding: identity User-Agent: python-urllib3/2.4.0 Content-Type: application/json Content-Length: 1498

{ "event": "updated", "timestamp": "2025-06-04T08:20:59.727826+00:00", "model": "site", "username": "password", "request_id": "b2f82525-6118-4427-9a40-659801d432ff", "data": { "id": 2, "url": "/api/dcim/sites/2/", "display_url": "/dcim/sites/2/", "display": "test_issue", "name": "test_issue", "slug": "test_issue", "status": { "value": "active", "label": "Active" }, "region": null, "group": null, "tenant": null, "facility": "", "time_zone": null, "description": "", "physical_address": "", "shipping_address": "", "latitude": null, "longitude": null, "comments": "test", "asns": [], "tags": [], "custom_fields": {}, "created": "2025-06-04T08:13:03.378951Z", "last_updated": "2025-06-04T08:20:59.675324Z" }, "snapshots": { "prechange": { "created": "2025-06-04T08:13:03.378Z", "description": "", "comments": "", "name": "test_issue", "slug": "test_issue", "status": "active", "region": null, "group": null, "tenant": null, "facility": "", "time_zone": null, "physical_address": "", "shipping_address": "", "latitude": null, "longitude": null, "asns": [], "custom_fields": {}, "tags": [] }, "postchange": { "created": "2025-06-04T08:13:03.378Z", "last_updated": "2025-06-04T08:20:59.675Z", "description": "", "comments": "test", "name": "test_issue", "slug": "test_issue", "status": "active", "region": null, "group": null, "tenant": null, "facility": "", "time_zone": null, "physical_address": "", "shipping_address": "", "latitude": null, "longitude": null, "asns": [], "custom_fields": {}, "tags": [] } } } Completed request #1

8. Create new event with condition

{ "id": 5, "url": "http://127.0.0.1:8000/api/extras/event-rules/5/", "display_url": "http://127.0.0.1:8000/extras/event-rules/5/", "display": "test_event_with_condition.", "object_types": [ "dcim.site" ], "name": "test_event_with_condition.", "enabled": true, "event_types": [ "object_updated" ], "conditions": { "op": "contains", "attr": "data.tenant.name", "value": "test_value", "negate": true }, "action_type": { "value": "webhook", "label": "Webhook" }, "action_object_type": "extras.webhook", "action_object_id": 3, "action_object": { "id": 3, "url": "http://127.0.0.1:8000/api/extras/webhooks/3/", "display": "test_issue_wb", "name": "test_issue_wb", "description": "" }, "description": "", "custom_fields": {}, "tags": [], "created": "2025-06-04T08:27:43.737571Z", "last_updated": "2025-06-04T08:30:54.429381Z" }

9. Edit created site, delete comment - “test”. Here is change log entry from change log:

"count": 65, "next": "http://127.0.0.1:8000/api/core/object-changes/?limit=50&offset=50", "previous": null, "results": [ { "id": 65, "url": "http://127.0.0.1:8000/api/core/object-changes/65/", "display_url": "http://127.0.0.1:8000/core/changelog/65/", "display": "DCIM | site test_issue updated by password", "time": "2025-06-04T08:20:59.686892Z", "user": { "id": 1, "url": "http://127.0.0.1:8000/api/users/users/1/", "display": "password", "username": "password" }, "user_name": "password", "request_id": "b2f82525-6118-4427-9a40-659801d432ff", "action": { "value": "update", "label": "Updated" }, "changed_object_type": "dcim.site", "changed_object_id": 2, "changed_object": { "id": 2, "url": "http://127.0.0.1:8000/api/dcim/sites/2/", "display": "test_issue", "name": "test_issue", "slug": "test_issue", "description": "" }, "prechange_data": { "asns": [], "name": "test_issue", "slug": "test_issue", "tags": [], "group": null, "region": null, "status": "active", "tenant": null, "comments": "", "facility": "", "latitude": null, "longitude": null, "time_zone": null, "description": "", "custom_fields": {}, "physical_address": "", "shipping_address": "" }, "postchange_data": { "asns": [], "name": "test_issue", "slug": "test_issue", "tags": [], "group": null, "region": null, "status": "active", "tenant": null, "comments": "test", "facility": "", "latitude": null, "longitude": null, "time_zone": null, "description": "", "custom_fields": {}, "physical_address": "", "shipping_address": "" } },

10. Check web hook receiver and make sure that you didn’t receive any changes for events: test_event_with_condition and test_issue_event_without_condition.

Here we see that there is no any request received only one that we have already received.

I've added try except to code just to show the place of exception:

Here is exeption.

Traceback (most recent call last): File "/Users/password/Documents/netbox-4.2.2/netbox/extras/events.py", line 91, in process_event_rules if not event_rule.eval_conditions(data): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/password/Documents/netbox-4.2.2/netbox/extras/models/models.py", line 143, in eval_conditions return ConditionSet(self.conditions).eval(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/password/Documents/netbox-4.2.2/netbox/extras/conditions.py", line 161, in eval return func(d.eval(data) for d in self.conditions) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/password/Documents/netbox-4.2.2/netbox/extras/conditions.py", line 161, in return func(d.eval(data) for d in self.conditions) ^^^^^^^^^^^^ File "/Users/password/Documents/netbox-4.2.2/netbox/extras/conditions.py", line 82, in eval result = self.eval_func(value) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/password/Documents/netbox-4.2.2/netbox/extras/conditions.py", line 116, in eval_contains return self.value in value ^^^^^^^^^^^^^^^^^^^ TypeError: argument of type 'NoneType' is not iterable

Expected Behavior

Event won't added to queue, don't brake all events in cycle or it can be validate somehow while creating event with such mistakes.

Observed Behavior

All events include an event with a condition containing an error, which runs in a loop after this problematic event with the same obj_type, and actions are not added to the queue as expected.

[jnovinger]

Thank you for opening a bug report, @ravenrs . Unfortunately, the information you have provided is not sufficient for someone else to attempt to reproduce the reported behavior. Remember, each bug report must include detailed steps that someone else can follow on a clean, empty NetBox installation to reproduce the exact problem you're experiencing. These instructions should include the creation of any involved objects, any configuration changes, and complete accounting of the actions being taken. Also be sure that your report does not reference data on the public NetBox demo, as that is subject to change at any time by an outside party and cannot be relied upon for bug reports.

[ravenrs]

@jnovinger Hi, trust me it's enough to reproduce, I do the same in Prod/Test/Dev enviroment. Only you need the same version and event with condition that contains error, but ok, I will explain all steps from scratch.

[ravenrs]

@jnovinger I've change description of issue, but it doesn't matter what to change site/device, all events with the same obj_type won't added to queue and will drop silence.

[jnovinger]

@jnovinger I've change description of issue, but it doesn't matter what to change site/device, all events with the same obj_type won't added to queue and will drop silence.

Thanks, I'll take a look at the updated STR as soon as I can.

[github-actions[bot]]

This is a reminder that additional information is needed in order to further triage this issue. If the requested details are not provided, the issue will soon be closed automatically.

[ravenrs]

@jnovinger please update status here

[ravenrs]

@jnovinger Hi, I've fixed it, could you assign issue to me, to submit pull request.

[jeremystretch]

Event won't added to queue, don't brake all events in cycle or it can be validate somehow while creating event with such mistakes.

I'm not sure what you're trying to say here, but this appears to be working as expected on NetBox v4.3.3. The event rule does not trigger because the constraint is invalid: attr should be tenant.name rather than data.tenant.name. However, other event rules still trigger successfully after a failure.

[ravenrs]

@jeremystretch Event doesn't return False and just skipped, it was a cause of TypeError. I've checked source of 4.3.3:

def process_event_rules(event_rules, object_type, event_type, data, username=None, snapshots=None, request_id=None):
    user = User.objects.get(username=username) if username else None

    for event_rule in event_rules:

        # Evaluate event rule conditions (if any)
        if not event_rule.eval_conditions(data):
            continue

        # Compile event data
        event_data = event_rule.action_data or {}
        event_data.update(data)

there is no "try except" in this cycle, so depending on the queue in this cycle, all subsequent events will be skipped. I can test it on my local 4.3.3, if you want, but I don't think it's necessary, I added pull request to avoid Type error, but I didn't add try exept to this cycle so theoretically the result will be the same if some error appears. Do you agree?

[jeremystretch]

@ravenrs if you test for the behavior I've described you'll find that my statement is accurate.

The root issue here appears to be that the TypeError exception raised by the eval_contains() method when passed a null value is being accidentally swallowed by the flush_events() handler. So, the first step it to fix that, and ensure that any exception unrelated to the import of a function are raised appropriately. This will result in the TypeError being uncaught, exposing the secondary bug.

The secondary bug is that an invalid attr definition can result in passing an unexpected type to the evaluator methods. In this case, we're passing None where an iterable is expected, but I'm sure there are others. The fix for this is to catch and appropriately log any TypeError, ValueError, or similar exceptions.

@ravenrs are you interested in pursuing this work?

[ravenrs]

@jeremystretch Yes, Im interested, I've implement workaround in my production environment, but i'm interested to have it in Upstream. I will check flush_events().

[ravenrs]

@jeremystretch Please check my approach to fix this issue and share your mind. thanks.

[ravenrs]

@ravenrs if you test for the behavior I've described you'll find that my statement is accurate.

@jeremystretch I've just tested it on the latest 4.3.3. Unfortunately, the bug is still there - the same thing happens in the function process_event_rules. The problem is the first event rule causes an exception inside the cycle. It drops the processing of next events with similar event_type/object_type. Please take into account that this is the reason I'd raised bug report, not the improper logging or insufficient error types in the flush_events()'s try-except.

accidentally swallowed by the flush_events()

It was by design, but it's not the reason to raise the bug report.

[15/Jul/2025 08:21:27] "GET /extras/image-attachments/?embedded=True&object_type_id=6&object_id=1&return_url=%2Fdcim%2Fsites%2F1%2F HTTP/1.1" 200 2907
[15/Jul/2025 08:21:28] "GET /dcim/locations/?embedded=True&site_id=1&return_url=%2Fdcim%2Fsites%2F1%2F HTTP/1.1" 200 4330
[15/Jul/2025 08:21:29] "GET /dcim/devices/?embedded=True&site_id=1&rack_id=null&parent_bay_id=null&return_url=%2Fdcim%2Fsites%2F1%2F HTTP/1.1" 200 8578
[15/Jul/2025 08:21:30] "GET /dcim/sites/1/edit/ HTTP/1.1" 200 178006
**Cannot import events pipeline extras.events.process_event_queue error: argument of type 'NoneType' is not iterable**

I do understand that more error types should be caught there for proper logging, but it doesn't fix the issue of dropped processing of events.

So, the first step it to fix that, and ensure that any exception unrelated to the import of a function are raised appropriately. This will result in the TypeError being uncaught, exposing the secondary bug.

You are right, there is a problem with improper error logging, but it is not the root cause of the issue I'd raised.

This will result in the TypeError being uncaught, exposing the secondary bug.

The TypeError was caught by not accurate exception handler.

So let's separate it to two issues:

1. Improper logging of other error types.
2. Events of similar event_type/obj_type don't get processed once this TypeError occurs. Despite the presence of proper logging in flush_events().

Proposed solution for issues: 1.1. Added few more possible error types to cover different type to flush_events function.

2.1. Return False instead of raising TypeError while the evaluating rules. This should prevent cycle crush which drops processing of next events with similar event_type/obj_type. 2.2. Add try-except with proper logging to cycle in process_event_rules, process_event_queue to prevent cycle interruption. 2.3. It will be good, despite it's unnecessary, to add condition check like it work with permission rule and constrains. What do I mean: you can't add constrains with non-existence attribute of object.

P.S.: In general. The root cause analysis in my company has identified that for new event rules we need to review event rule conditions on the NetBox test server to prevent one event from breaking the processing of other events (for the same object/event type).

[jeremystretch]

@ravenrs I'm sorry but I don't have time to discuss this any further. It seems the guidance I provided above was insufficient, so I'm happy to take ownership of this issue myself.