Efficient Techniques for Tracking Graph Changes in Apache AGE

[MatheusFarias03]

Hello! There's a question on AGE's discord server related to graph changes and how to track them. I wanted you guys opinion first so that I can give the right answer there.

---

Question from Discord User: User: Murty Devarakonda Context: Apache AGE Discord Server

Hi there! I am pretty new to Apache AGE (1 day old 🙂 ) and ran into a very general question. Let's say I created a graph with my initial set of data. After some time, I bring in additions/deletions on the vertices and edges in the existing graph and updated the DB. How do I keep track of what changed in the graph after the update from the previous state of the graph? Generally, what are the techniques people follow for this in AGE? An immediate thing that comes to my mind is time-based audit log entries to keep track of the changes. On second thought, marking the edges and vertices that are impacted with a property (color coding??) is another thing. Need some suggestions on how to do this efficiently.

[jrgemignani]

@MatheusFarias03 This is a topic that we've been discussing internally -

Audit logs should already exist for any database action that can be logged, provided the DBA or admin for said database enables them. This would, of course, be the more raw (PG) data for the additions, modifications, and deletions from the graph database.

However, it might be worthwhile to look into adding options that would give log outputs that would be more graph centric and easier to understand.

[danielwambo]

Maintaining a time-stamped audit log is a common and effective way to track changes. Whenever a vertex or edge is added, modified, or deleted, record the change with a timestamp. This allows you to reconstruct the graph's history and understand how it has evolved over time. You can also include information about the user or system responsible for the change. This method is essential for auditing and compliance.

Also versioning and snapshots is another way. i.e saving a copy of the entire graph on different times.

[namsi-lia]

Storing metadata alongside graph elements might be beneficial in that this will store information about when the node and edges were created and when last they were modified.

Also incorporating triggers can also help to capture and handle graph changes in real-time. This will be able to capture events such as INSERT,UPDATE,DELETE operations happening in the graphs.

[MatheusFarias03]

Thank you for the awesome detailed answers @danielwambo and @namsi-lia ! Based on your comments, I believe you guys definitely have a deeper knowledge about this. As @jrgemignani mentioned, it might be interesting to add options for the log outputs to be more related to the graph, instead of the raw PG data. Would be nice if you guys implemented such a thing. What do you say?

[jrgemignani]

@MatheusFarias03 One would need to research the mechanism that PG uses to log this data and then apply it to AGE. It "could", emphasis on "could", be a relatively easy thing to add. I would imagine that there are built-in functions or macros to do this.

[danielwambo]

@MatheusFarias03 A good way to do this is by creating a separate table then store the log details in it i.e in any modification addition or deletion of vertices and edges. 1.First we Create the table CREATE TABLE Audit_logs( //here we can store audit information like timestamp,type of change(addition,deletion etc), changed value, id , new value); 2.Then we 'INSERT VALUES' to that table during any change on the graph INSERT INTO Audit_logs (timestamp, type_of_change, changed_ , change_id) VALUES (now(), 'add', 'vertex', 'value');

And more information about the changes can be added as fields on these logs.

[github-actions[bot]]

This issue is stale because it has been open 45 days with no activity. Remove "Abondoned" label or comment or this will be closed in 7 days.

[github-actions[bot]]

This issue was closed because it has been stalled for further 7 days with no activity.