foxsec-pipeline

foxsec-pipeline copied to clipboard

We should write tooling to support integration tests between the cloud functions in contrib/ and the pipeline code. Some examples: * Test that Guardduty findings get processed through Gatekeeper, get...

ajvb

Within our parsing logic, we make use of the pubsub timestamp rather than the parsed events timestamp ([`Parser.stripStackdriverEncapsulation`](https://github.com/mozilla-services/foxsec-pipeline/blob/master/src/main/java/com/mozilla/secops/parser/Parser.java#L259)). This creates problems in the case of old messages getting backfilled into...

ajvb

Currently cfgtick contains runtime options and transform documentation; it would be good in some cases to support the addition of arbitrary text blocks (potentially via configuration options) as well

ameihm0912

Update nginx parser to support parsing raw nginx logs

1

The parser currently only supports processing nginx log data in the form of a Stackdriver jsonPayload entry. This should be expanded to also support raw nginx log lines (either in...

ameihm0912

#305 adds timeouts to the iprepd writer IO transform, but they are hardcoded. These should be configurable with reasonable defaults.

ameihm0912

API: https://help.papertrailapp.com/kb/how-it-works/search-api/ Some past work in this area (Lua): https://github.com/mozilla-services/lua_sandbox_extensions/tree/master/papertrail/sandboxes/heka/input

ameihm0912

Look into alerting subnet summarization

1

Where addresses within alerts can be reasonably correlated to belonging to the same subnet, for example the same /24, add support to alerting output to potentially generate a secondary alert...

ameihm0912