plugins icon indicating copy to clipboard operation
plugins copied to clipboard

new(anomalydetection): Initial Scope - CountMinSketch Powered Probabilistic Counting and Filtering

Open incertum opened this issue 1 year ago • 1 comments

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area plugins

/area registry

/area build

/area documentation

What this PR does / why we need it:

Introduce a new anomalydetection plugin, as outlined in the Proposal.

Which issue(s) this PR fixes:

Fixes https://github.com/falcosecurity/falco/issues/3117

https://github.com/falcosecurity/falco/issues/3117

Fixes #

Special notes for your reviewer:

Sharing some hopefully useful notes:

  • Check out the README first, as it summarizes current limitations and the initial scope
  • Much of the code was duplicated from falcosecurity/libs; I’ve indicated this to show that this code doesn’t need review here. If you find issues, let’s address them in libs instead
  • Hot reloading seems to be working (no segfaults observed), except toggling enabled/disabled during a hot reload doesn’t work, which we can likely ignore
  • The project builds in a container suitable for building Falco, so we can set up CI
  • Initial local tests confirm the intended functionality, including the reset timers
  • Unit tests cover the most critical functions but could still be improved
  • cms.h contains the algorithm/math (it's probabilistic counting only, not AI/ML); the rest is mostly generic plugin setup or configuration.
  • Config initialization is designed to error out if the plugin is used for currently unsupported cases
  • The profile field filtercheck code might be removed if libs filterchecks are opened to plugins, which would make it more stable and less risky. Additionally, translating Falco filtercheck logic to the plugin framework was very tedious
  • Further testing on more robust servers is needed to check for performance and stability issues, such as CPU spikes or segfaults after extended runs
  • Most things had to be wrapped in try-catch statements, as simple checks for nullptr were not possible ...

For this first version, I am primarily concerned about performance and stability (no segfaults). I would truly appreciate a very critical review in this regard. Thanks a bunch in advance!

incertum avatar Feb 27 '24 06:02 incertum

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: incertum Once this PR has been reviewed and has the lgtm label, please assign mstemm for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

poiana avatar Jul 25 '24 00:07 poiana