flowpipeline
flowpipeline copied to clipboard
bwNetFlow
Process network flows using fully configurable pipelines.
Allows to buffer flows in memory and (additionally) on disk in zstd compressed files. Add a small segment (filegate) for testing purposes to pause the pipeline when a file exists.
/flowdata metrics don't seem to be most reliable. Direction label always indicates "Incoming" (regardless of "- segment: remoteaddress" configuration and actual state; btw, documentation is a bit vague on this...
enable lists inside of segments in configuration. e.g. ```yml - segment: example config: list: - val1 - val2 ``` This should be extended by modifing https://github.com/bwNetFlow/flowpipeline/blob/master/pipeline/config.go
This can be achieved using stdout and stdin segments, but database readers would be nice to have. Also, think about timing of messages, respect intervals from data or just send...
- [ ] rename the segment - [ ] rename the target fields (goflow2 uses `InIf` and `OutIf`, we're still using `SrcIfDesc` and so on - [ ] allow synchronous...
## Example ``` - segment: prometheus # collect and export peering statistics config: endpoint: ":8080" labels: "SrcAS,DstAS,SrcIfDesc,DstIfDesc,SamplerAddress,FlowDirection" ``` ## Problem The prometheus segment will add every combination of labels it...
I modified a json export with another tool (jq) and was unable to read it back into flowpipeline. In the first attempt, jq pretty-printed the json with newlines between each...
Hello, I am evaluating this flowpipeline in my environment. I want to collect netflow from several DataCenter devices like Firewalls, Loadbalancer and Routers. I want to collect these logs using...
Just a first draft. Effectively, this spawns `j` concurrent pipelines that are wired up separately. Hence, it might affect the order of throughput. We could leverage concurrency even better my...