CrackMapExec icon indicating copy to clipboard operation
CrackMapExec copied to clipboard

[RFF] JSON output logging

Open aress31 opened this issue 4 years ago • 23 comments

Right now parsing result outputs from CME is not really intuitive and requires a lot of crafty bash one-liners.

To address this issue and improve on the tool, how about offering JSON output (with --json) to allow for easily parsing (e.g. extract hosts, users, passwords, shares) with the help of jq and ConvertFrom-JSON.

aress31 avatar Aug 23 '21 20:08 aress31

I really really would this option as well. It does exactly what we need, but I need to enrich the data with our CMDB. To do that I need to parse the data somehow. JSON output would be much appreciated.

sandervandegeijn avatar Mar 01 '22 08:03 sandervandegeijn

Would be a great feature +1

SmashRocket avatar Mar 01 '22 08:03 SmashRocket

Is there any update on this? I would like to be able to properly parse the output data from the CME scripts but there isn't really a way to efficiently do this.

Senne-s avatar Mar 20 '23 12:03 Senne-s

The sponsors version has a log feature where everything is send to a file.

mpgn avatar Mar 20 '23 12:03 mpgn

Alternately you can export the data from cmedb as csv if the usual log output doesn't suite you

NeffIsBack avatar Mar 20 '23 12:03 NeffIsBack

I'm also interested in JSON output.

daanwa avatar Apr 13 '23 12:04 daanwa

@Marshall-Hallenbeck maybe a feature for the new cmedb? Is there an easy way to export as json?

NeffIsBack avatar Apr 13 '23 13:04 NeffIsBack

@aress31 @ict-one-nl @SmashRocket @Senne-s @daanwa Do you mean the entire output (so instead of the normal output, it's all JSON), or just an option to have normal STDOUT prints and JSON to a file, or a way to dump CMEDB information as JSON? Could you give some examples and I can see what I can put together to fit the needs.

Use cases on how you're interacting with results, and how it would look with JSON would be appreciated.

Marshall-Hallenbeck avatar Apr 13 '23 14:04 Marshall-Hallenbeck

I want the results of the tool to be incorporated into our tooling. However, the current output shown on STDOUT is somewhat cumbersome to parse. So in my case it would be most handy to have a command line flag that outputs the results in JSON to a file.

Preferably I would not have to use the cmedb cli and I could just provide the JSON output flag to the cme app.

daanwa avatar Apr 13 '23 17:04 daanwa

@daanwa Thanks for that information. It might be pretty hard to do, but I can look into it. Can you provide an example of how you would expect the JSON structure to look like with several hosts & modules?

Marshall-Hallenbeck avatar Apr 13 '23 18:04 Marshall-Hallenbeck

Yup, I'd like to parse the output with something like a python script to write to opensearch and enrich the info with contents of our CMDB :)

sandervandegeijn avatar Apr 13 '23 19:04 sandervandegeijn

i'm not sure to see the link between crackmapexec and opensearch

mpgn avatar Apr 13 '23 21:04 mpgn

That's up to my scripting skills (to ingest data into our SIEM and generate reports). But to do anything useful I need to interact with the data, for that I'd need parsable output; json works fine for that :)

sandervandegeijn avatar Apr 13 '23 21:04 sandervandegeijn

@ict-one-nl What sort of structure would you expect as output? Can you provide an example?

Marshall-Hallenbeck avatar Apr 14 '23 14:04 Marshall-Hallenbeck

Hmm, doesn't really matter, I can image a header like object for the general stuff and a list of objects for each line outputted by CME containing all the details for that line?

sandervandegeijn avatar Apr 19 '23 17:04 sandervandegeijn

Agree with @ict-one-nl, doesn't matter a whole lot as long as it is consistent. But I can imagine something such as the JSON below (depending on which command you run of course):

{
  "smb": [
    {
      "host": "hostname or ip",
      "user": "username",
      "password": "password",
      "domain": "domain",
      "status": "STATUS_LOGON_FAILURE"
    }
  ]
}

Potentially with some additional keys that make it easier to determine if the command is considered successful, i.e. when we receive a STATUS_LOGON_FAILURE there is an extra key of "login": false or something.

daanwa avatar Apr 20 '23 12:04 daanwa

but what's the point ? I don't understand

mpgn avatar Apr 20 '23 13:04 mpgn

Why you would want a parseable format? Just like nmap output; I can use it to create reports, cross reference with the CMDB, do lookups for owners of systems in our databases, write a text file with HEY THIS SHARE IS OPEN!!, make it searchable in a tool like elastic, put in our SIEM, etc. Possibilities are endless. This is not only about targeting and exploiting one host, it's also about doing recon, integrating with other systems and providing an overview. It makes finding vulnerable hosts much much easier.

sandervandegeijn avatar Apr 20 '23 14:04 sandervandegeijn

Exactly. It is mainly aimed at integrating with some of our other tooling or for collecting additional authentication information about certain data points.

@mpgn Perhaps there is already a different way that can do this and I'm just not aware of it? Currently I'm parsing the sqlite databases but I've seen that not all commands/modules populate the db's.

daanwa avatar Apr 20 '23 14:04 daanwa

Using the SQLite db quite cumbersome for what's needed in this case. Just a different formatter for the data that's already there would suffice :)

To give some more context, I'm working at a university in the Netherlands on the blue team. Example use case: I want to use this to scan our ±16k hosts to find vulnerable stuff and open shares that contain data that shouldn't be there. I'll probably combine masscan with CME to go through the network quickly. It's also about chaining tools :) Extensibility is nice :)

sandervandegeijn avatar Apr 20 '23 14:04 sandervandegeijn

Okay, did it the sqlite way for now:

sqlite3 -json /root/.cme/default/smb.db 'select * from shares inner join computers on computers.hostname = shares.computerid where read = true and name NOT LIKE "%print%"' > /root/.cme/output.json

sandervandegeijn avatar Apr 23 '23 19:04 sandervandegeijn

So, my 2 cents, if we add a json option it should be used in combinaison with --log file option.

cme <proto> <ip> -u <user> -p <pass> --log filename --json

And the output in the log file 'filename' will be in json. I know @NeffIsBack already work on this, I think this is the best way to do it.

mpgn avatar May 02 '23 15:05 mpgn

Yeah that would work for me!

daanwa avatar May 04 '23 08:05 daanwa