PyMISP icon indicating copy to clipboard operation
PyMISP copied to clipboard

Feature request: Feed generator option to continue an existing feed

Open iglocska opened this issue 7 months ago • 2 comments

Currently the feed generator will take a snapshot of all data matching the parameters, generate the JSON representations and a manifest.

But for purposes of using the feed generator as a "cold storage" solution, it would be great if we could run it in an append mode where the manifest gets any event being dropped added.

iglocska avatar May 22 '25 14:05 iglocska

Something like checking the manifest, taking the last update timestamp in the last event inserted and searching from that time?

If an old event is updated, the response to the search query returns the complete event, right?

Rafiot avatar May 22 '25 18:05 Rafiot

Something like checking the manifest, taking the last update timestamp in the last event inserted and searching from that time?

If an old event is updated, the response to the search query returns the complete event, right?

Yup, that's correct. The idea would be to only add/update what has changed/was added and keep the previously generated events intact (unless there were modifications of course). This would allow users to clean up older events on their instances.

iglocska avatar Jun 03 '25 11:06 iglocska