Add option to output events to a file

[ayoubfaouzi]

Helllo @rabbitstack,

Scanning over the doc, I was not able to find if we can output the events to a JSON file.

I see that we have a console sink, and I can turn the results to JSON, do I have to redirect the stdout to a file to get the results stored in a file ?

I don't mind redirecting stdout to a file, I am just facing an issue where I can't turn off the PE module. I have the pe.enabled=false in my yaml config, but I am still seeing such errors in the console:

As a result, when later I try to parse the JSON lines, it cannot be de-serialized properly because it has non JSON entries on it.

Btw, I see that you are running an older version of the pe module, I can definitely fix that error if it happens on the last version, I am just not sure which PID or hash that causes it.

I am impressed how many great new changes has been added to this tool since I have used it last time. Good work mate 🚀

Cheers.

[rabbitstack]

Hi @LordNoteworthy,

I apologize for the delay :). A better approach would be using the captures. The capture is essentially a binary file containing the full state of the system processes along with the events. Then you can use a console output with the JSON formatter to further transform the events. I reckon the latest Fibratus version suffers from the issue that impedes capturing the events, so you'll have to install 1.10.0 until 2.2.0 is out where the fix will land.

The pe module is used in many other places asides from the PE metadata, that's why you are seeing those errors. I would really love to have those fixed :).

Thanks for the nice words mate. Stay tuned for the next release which will be a game changer!

[ayoubfaouzi]

Thanks for your return 👍.

In my use case, I want to definitely ignore all system processes, and I am specifically watching only certain processes. I will give it a try though.

I will have a look into why the PE module is producing those error messages, I just don't know how can I make Fibratus tell me which image file is causing those parsing errors.

Cheers.

[rabbitstack]

Hi @LordNoteworthy ,

What I really meant by system processes is that the capture file will contain the snapshot of all running processes including their extended metadata. This information is used to fully enrich every event with its corresponding process.

pe is extensively used by Fibratus. I'll point out exact locations in the code if you want to dig further.

P.S. sorry for late replies. I'm currently not at my usual location

[rabbitstack]

Hi again, @LordNoteworthy

The stable Fibratus version utilizes the PE parser in the following code areas:

• image processor for verifying the executable/DLL digital signature
• PE filter accessor. You can probably ignore for your investigation if you're not using rules or CLI filters
• PE enricher, which is disabled by default

Hope this helps

[rabbitstack]

Hey @LordNoteworthy,

https://github.com/rabbitstack/fibratus/pull/352 should get rid of most of the error log messages that you spotted. In addition, #350 bumps saferwall/pe to the latest version.

Separately, all capture-related issues have been addressed in Fibratus 2.2.1, if you want to give it a try.

[ayoubfaouzi]

Amazing ! I will try it during the week,