Argus icon indicating copy to clipboard operation
Argus copied to clipboard

Performance issues with many metadata change events

Open elfjes opened this issue 9 months ago • 0 comments

Updating an incident's metadata currently writes a full dump of the old and the new metadata to the event description, which may lead to a (very) large event description field if the metadata object is large. By itself this leads to event descriptions that are so verbose that they become unreadable.

When combining this with incidents that have many events, this significantly hinders performance. I ran into a situation where we had +/- 300 incidents and >20000 events, our argus instance became pretty much unusable. Updating an incident would use so much memory that our gunicorn workers were killed. Looking at what queries are run it shows that all events for an incident are fetched when updating that incident:

SELECT "argus_incident_event"."id", "argus_incident_event"."incident_id", "argus_incident_event"."actor_id", \
"argus_incident_event"."timestamp", "argus_incident_event"."received", "argus_incident_event"."type", \
"argus_incident_event"."description" FROM "argus_incident_event" WHERE "argus_incident_event"."incident_id" = <incident-id> \
ORDER BY "argus_incident_event"."timestamp" DESC

This seems strange because the incident update only needs to create a new event, not retrieve every other event related to that incident I'd think

I'd be very grateful if you could provide the following fixes:

  • do not dump all of the metadata in the description of the incident. I'd suggest that an "Updated metadata" would be sufficient
  • figure out why updating an incident triggers selecting all previous events for that incident from the database

elfjes avatar May 20 '24 13:05 elfjes