tracee icon indicating copy to clipboard operation
tracee copied to clipboard

Streamline events pipeline

Open NDStrahilevitz opened this issue 3 years ago • 1 comments

Prerequisites

  • [x] Is this an EPIC issue describing other issues ? (add label: EPIC)
  • [ ] I'm going to create a PR to solve this (assign to yourself)
  • [ ] Someone else should solve this.

Feature description

This an epic which may include bug fixes and features, but probably mostly refactors.

Context

Currently, the ebpf package is structured to use a pipeline, where channels are passed to pipeline function which return other channels:

image

However, there is no formal abstraction for this besides a convention set by the code. Some problems that arise from this, for example:

  1. The output buffers may vary in size between the pipeline functions.
  2. Pipeline function signatures may change between pipeline stages
  3. The pipeline passes a trace.Event and a gocontext, however we may require additional defined context between pipeline steps

I'm suggesting to refactor this part as part of the ongoing "stabilization" effort, and possibly move the pipeline to it's own pipeline package, which can then be used alongside a separated events package.

In addition, the pipeline should be able to easily process events from various eBPF buffers (for example the network event buffer)

Additional

PR: #1689 (pipeline: split process and sink stages)

NDStrahilevitz avatar Apr 25 '22 11:04 NDStrahilevitz

Related - #1922

NDStrahilevitz avatar Jul 06 '22 13:07 NDStrahilevitz