falco icon indicating copy to clipboard operation
falco copied to clipboard

Access to more variables in output_fields

Open wdoekes opened this issue 3 years ago • 22 comments

Motivation

Right now, we're parsing the jsonOutput: true json generated by falco and feeding it to Slack.

Instead of using the unformatted "output", we use the "output_fields". This way, we can do something like this.

We take this json:

{
  "output": "14:33:48.105952346: Error File below / or /root opened for writing (user=root user_loginuid=-1 command=touch /wjd-test parent=runc:[1:CHILD] file=/wjd-test program=touch container_id=c0a8e388a8c2 image=falcosecurity/falco) k8s.ns=kube-falco k8s.pod=falco-zjsqj container=c0a8e388a8c2",
  "priority": "Error",
  "rule": "Write below root",
  "source": "syscall",
  "tags": [
    "filesystem",
    "mitre_persistence"
  ],
  "time": "2022-07-12T12:33:48.105952346Z",
  "output_fields": {
    "container.id": "c0a8e388a8c2",
    "container.image.repository": "falcosecurity/falco",
    "evt.time": 1657629228105952300,
    "fd.name": "/wjd-test",
    "k8s.ns.name": "kube-falco",
    "k8s.pod.name": "falco-zjsqj",
    "proc.cmdline": "touch /wjd-test",
    "proc.name": "touch",
    "proc.pname": "runc:[1:CHILD]",
    "user.loginuid": -1,
    "user.name": "root"
  }
}

And we turn it into this:

image

We can create that nicely formatted message because all fields are separately accessible.

output: "File below / or /root opened for writing (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline parent=%proc.pname file=%fd.name program=%proc.name container_id=%container.id image=%container.image.repository)"

All %-fields mentioned in the output are available in output_fields, as seen above.

But, we do not have access to -- for instance -- proc.aname[2].

The easy solution sounds like amending the output: in the default rules. But alas, this does not work.

    - rule: Write below root
      output: "File below / or /root opened for writing (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline parent=%proc.pname file=%fd.name program=%proc.name container_id=%container.id image=%container.image.repository)"
      append: true
      condition: >
        and not (expected_write_below_root_behaviour)

The conditions are amended, but the output is not amended/overwritten: so I cannot get more values in the output_fields without copying the entire Write below root rule to something new.

Feature

I'd like to be able to get more values in the output_fields.

  • This could be by overwriting the rule name and adding them in the output,
  • or by setting a separate also_add_these_variables_in_output_fields: [proc.aname[2], ...] on the rule,
  • or through some other means.

Cheers, Walter Doekes OSSO B.V.

wdoekes avatar Jul 12 '22 12:07 wdoekes

This makes a lot of sense. Will look into it and hopefully get this in the next release!

/milestone 0.33.0

jasondellaluce avatar Jul 12 '22 13:07 jasondellaluce

I just noticed the same thing. Parsing output_fields from k8s_audit alerts and wanted to add the source ip of the request to all alerts.

- rule: Attach/Exec Pod
  append: true
  output: src_ip=%json.value[sourceIPs/0]

- rule: K8s Namespace Deleted
  append: true
  output: src_ip=%json.value[sourceIPs/0]
...

Falco 0.32.2 fails to load it because it does not contain an exception or condition property

Error: Could not load rules file /etc/falco/rules.d/k8s_audit_rules.z.local.yaml: 1 errors:
Appended rule must have exceptions or condition property
---
- rule: Attach/Exec Pod
  append: true
  output: src_ip=%json.value[sourceIPs/0]
---

Would be great to see support for appending to output or even just adding extra fields to output_fields in 0.33.0 😃

robinlandstrom avatar Aug 18 '22 13:08 robinlandstrom

Out of curiosity, was this supported in Falco <= 0.31.1, or is this a brand new feature request? I agree this can be useful for many.

@robinlandstrom Would you like to open. a feature request issue on this repo to support the cases you reported in these code examples?

jasondellaluce avatar Aug 22 '22 09:08 jasondellaluce

I don't think it was supported in Falco <= 0.31.1, have not tried thou.

Just being able to append to rule output as shown by @wdoekes would be enough for my use case. If condition is required one can always just add condition: and always_true, but sure I think it would be a sane default if the condition is missing from a rule append.

robinlandstrom avatar Aug 23 '22 10:08 robinlandstrom

/milestone 0.33.0

jasondellaluce avatar Aug 23 '22 13:08 jasondellaluce

/remove-milestone 0.33.0

/milestone 0.34.0

jasondellaluce avatar Oct 03 '22 07:10 jasondellaluce

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

poiana avatar Jan 01 '23 09:01 poiana

/remove-lifecycle stale

jasondellaluce avatar Jan 04 '23 11:01 jasondellaluce

/milestone 0.35.0

jasondellaluce avatar Jan 10 '23 17:01 jasondellaluce

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

poiana avatar Apr 10 '23 19:04 poiana

/remove-lifecycle stale

Andreagit97 avatar Apr 11 '23 12:04 Andreagit97

/milestone 0.36.0

Sorry we havent' got the time to work on this during this release cycle.

FedeDP avatar May 29 '23 09:05 FedeDP

In addition to defining extra output fields for a single rule, it would be nice to be able to define extra output fields for every rule with a certain tag (e.g. k8s) or source (e.g. k8s_audit).

sboschman avatar Jun 28 '23 10:06 sboschman

@wdoekes @sboschman @robinlandstrom I am on it. There are other issues remotely realted to this, please allow for some time to triage them all and then decide on necessary patches to support all features requests. The first PR is just to expand append mode option to output and tags.

incertum avatar Jul 05 '23 21:07 incertum

@incertum if you want more details on the setup we are trying to run, to perhaps better decide if and how falco can better support such a scenario, let me know. It is of course just one of many scenarios described in this issue and all the related issues, but hopefully there is enough similarity to cover most/all of them.

sboschman avatar Jul 07 '23 15:07 sboschman

@sboschman this would be amazing! And agreed let's collect all information. We can then see what could be the best solution for https://github.com/falcosecurity/falco/issues/2127#issuecomment-1611143944.

By the way my setup is a custom script and I do exactly what you describe in there I add certain output fields when xyz tags are available. But of course I create a whole new patched up rules file where I also prioritize the ordering of rules given it's currently "first match wins". This is also something @jasondellaluce and @leogr are thinking about a lot how we can expose better options to end users.

Thank you!

incertum avatar Jul 07 '23 15:07 incertum

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

poiana avatar Jan 12 '24 09:01 poiana

Sorry, I had lost track of this. /remove-lifecycle stale

cc @jasondellaluce

leogr avatar Jan 12 '24 11:01 leogr

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

poiana avatar Apr 11 '24 15:04 poiana

This issue should be addressed by https://github.com/falcosecurity/falco/pull/2981 cc @LucaGuerra correct? :thinking:

leogr avatar Apr 12 '24 14:04 leogr