PyMISP
PyMISP copied to clipboard
MISP
Add new feature to delete list of attributes by ID
Request a PyMISP method that accepts a list of attributes to be deleted. In version 2.4.133 api.py exposes method delete_attribute(id) which works fine. If run on the MISP server host the rate is about 22 per second, which is tolerable. Still it seems silly to POST 100,000 or other horribly large number of times when I could POST once. FWIW, we would like to use this feature to clean the mess of soft-deleted attributes left by the delta-merge algorithm on fixed-event feeds.
I'm hoping this is trivial bcos apparently the MISP server supports this already according to this comment thread; it requires an event ID (altho the delete_attribute method does not), but that doesn't seem hugely onerous:
https://github.com/MISP/MISP/issues/6010#issuecomment-660891990
What's the right way - extend the delete_attribute method to accept a list or add a new method? If you will please reply with the team's preference then I will make a proposal for a change to api.py. Thanks for listening.
I have to correct what I wrote above. The rate for deleting attributes from an event seems to vary with the number of attributes on the event. A PyMISP script can delete multiple attributes per second from an event with a small (fewer than 1,000) attributes. But if the event has 100K attributes, a delete operation can take 5..10 seconds. I don't know what is responsible for the large execution-time cost, if it's locking, queries or whatever. I hope the CIRCL team can advise if a single delete-id-list function would be faster than successive REST calls.
Yeah, this method is meant for the UI, not for the API, we need to implement something specific for the API. My guess regarding the time is that as it deletes attributes sequentially, and each deleted attribute triggers an update in the correlation table.
Well maybe if the team selects this MISP extension for development, the PyMISP feature will be easy: https://github.com/MISP/MISP/issues/6934
Related... https://github.com/MISP/MISP/issues/6934#issuecomment-1847508251
@iglocska This is an important use case for us. Let us know if we can provide any addition info to help move this feature request forward. Thanks.
@iglocska Hi... revisiting this with 2.4.191 -- any chance that we can see this feature sometime soon?