Aging of Persistency

[skopikf]

trafficstars

If the system, whose logs are consumed and investigated, changes then once learned values (e.g., IP addresses, urls, user names, process names, ids etc.) remain in the persistency forever. The AMiner then becomes too permissive as values that should not occur any more are still allowed (e.g., in case of authentication logs: if a user is deactivated and not used any more, he remains in the persistency and if reused by an attack even months or years later, it would still be a perfectly fine and allowed value).

Additional parameters to capture when a once learned value occured the last time and subsequent deletion of values that did not occur for a configurable period of time, could be a solution. The default values could be set to 0 or -1 resulting in the same behavior as right now (values remain in the persistency forever) to be backward-compatible.

Having an aging feature would also allow to run the AMiner continuously in learning mode, so alerting on the first occurence of any anomaly would still occur, but making it then part of the persistency. As this value would age out anyway, if it was indeed an anomaly, the next occurence would be triggered again as anomaly. In case the anomaly happens more often, with higher frequency then the set aging, then the anomaly is actually "normal". A real-world use case could be user logins: Continuously monitor which users login and alert if there are new users AND alert if a user wasn't logged in for a defined period of time (this is a user that has ageed out). This is a pretty standard SIEM case. Same could be used for programs used on a laptop (names of created processes). Those that occur on an almost daily basis become part of the persistency and remain there. Also after every major OS and Appliacation update new values need to be learned quickly. If a program is not used any longer it ages out, and we will not maintain thousands of dead entries (e.g. from old versions of the same program) in the persistency.