New Idea for an easier tool to inventory all executables

[dremspider]

I don't see an easy way to set up the initial information about executables on your system. I played with a Python script (sorry my ruby skills are limited) that parses the log file and grabs unique paths and developers that have been launched. My idea is to extend this to allow the user to let this run for a while, and then run this tool that would parse through the logs and allow them to add exceptions as they would see fit. Right now, it is just dumping out the info.

More ideas for the future: -Dump to some sort of json/xml structure and allow the user to edit, user can then accept all executables in that xml structure -Allow the user to see what recently has been denied, then maybe approve the executable easily -Allow the user to only see non-signed executables and signed executables (so that they can accept the cert first, then add a rule for the path

Is there any tool like this? Would you be interested in me extending this idea? Should I just make it a separate project or would you be interested in integrating this with your project?

The output is this:

Unique Executables: Path of the executable: /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mdworker Common name of the certificate that signed the executable: (null) SHA256 hash of the executable: None SHA256 hash of the cert that signed the executable: None

 Path of the executable: /Users/dlohin/Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle/Contents/Resources/GoogleSoftwareUpdateAgent.app/Contents/MacOS/GoogleSoftwareUpdateAgent
     Common name of the certificate that signed the executable: Developer ID Application: Google Inc.
     SHA256 hash of the executable: a71d1105b521135c071b34c3912f9e1fb7dcd14efcb4c699d27838c977294c20
     SHA256 hash of the cert that signed the executable: 15b8ce88e10f04c88a5542234fbdfc1487e9c2f64058a05027c7c34fc4201153

 Path of the executable: /bin/ps
     Common name of the certificate that signed the executable: Software Signing
     SHA256 hash of the executable: 0b0f518e945e2803772a85dcd48366955abf345b2cd1cbc24b2004aab70e94e7
     SHA256 hash of the cert that signed the executable: 2aa4b9973b7ba07add447ee4da8b5337c3ee2c3a991911e80e7282e8a751fc32

 Path of the executable: /Applications/Google Chrome.app/Contents/Versions/45.0.2454.93/Google Chrome Helper.app/Contents/MacOS/Google Chrome Helper
     Common name of the certificate that signed the executable: Developer ID Application: Google Inc.
     SHA256 hash of the executable: 1d8d25946e09aecb63fb683e019914db235540d5ae80a925dda9092b19a13411
     SHA256 hash of the cert that signed the executable: 15b8ce88e10f04c88a5542234fbdfc1487e9c2f64058a05027c7c34fc4201153

 Path of the executable: /Applications/GPG Keychain.app/Contents/MacOS/GPG Keychain
     Common name of the certificate that signed the executable: Developer ID Application: Lukas Pitschl
     SHA256 hash of the executable: 4a9d7ef5f8ff93f346802ec5d541e7fbf002a7ba3eb5d5464ce69ff32bc56329
     SHA256 hash of the cert that signed the executable: cc4c6b01ade3162ee5d3aa46139efe89f210767edc9227a8185df8d3b1906bf0

 Path of the executable: /sbin/umount
     Common name of the certificate that signed the executable: (null)
     SHA256 hash of the executable: None
     SHA256 hash of the cert that signed the executable: None

 Path of the executable: /Applications/Microsoft Word.app/Contents/MacOS/Microsoft Word
     Common name of the certificate that signed the executable: Developer ID Application: Microsoft Corporation (UBF8T346G9)
     SHA256 hash of the executable: 8fa785826445d9b3e86d37f53d101639431ded8bc763f1f0321b1d5ca6f1fb0e
     SHA256 hash of the cert that signed the executable: adced5ef3635ab1debda86f02dc55e96cefedf7bd7fe7519f0a806092bf75873

 ...CONTINUED......

Unique Publishers: Common Name: Apple Mac OS Application Signing SHA256 hash of the cert that signed the executable: ad9dba778a3a1204a02298193be3c8d184a6bdc0d10ae109dfaf356722b3cf92

 Common Name: Developer ID Application: Google Inc.
 SHA256 hash of the cert that signed the executable: 15b8ce88e10f04c88a5542234fbdfc1487e9c2f64058a05027c7c34fc4201153

 Common Name: Software Signing
 SHA256 hash of the cert that signed the executable: 2aa4b9973b7ba07add447ee4da8b5337c3ee2c3a991911e80e7282e8a751fc32

 Common Name: Developer ID Application: Wireshark Foundation, Inc. (7Z6EMTD2C6)
 SHA256 hash of the cert that signed the executable: 9f9dd26b6ad69cd86438ee416d527b8a9ed871a2bd75cdf8f9cf2345a9cbeb38

....Continued......

Code is:

SantaParseLibs.py:

author = 'dlohin'

class santa_data_structure(object): def init(self,executable_cn = None, executable_hash = None, cert_hash = None): """ Pass in each entry of santa.log and add it to a dictionary :type executable_cn: str :type executable_hash: str :type cert_hash: str """ self.executable_cn= executable_cn self.executable_hash = executable_hash self.cert_hash = cert_hash

SantaParser.py:

author = 'dlohin'

from SantaParseLibs import santa_data_structure import re

f = open('/var/log/santa.log',mode='r')

unique_executables = {} unique_publishers={}

for line in f: #print(i) exectuble_info = line.split("|") executable_dict = {}

#Hit each variable in the log file
for i in exectuble_info:
    info = i.split("=")
    #We have some stuff about service starting in the beginning, ignore it
    try:
        executable_dict[info[0]] = info[1]
    except:
        pass

if executable_dict:
    #Already have the executable, ignore
    if executable_dict['path'] not in unique_executables:
        #see if it is a signed executable first
        if 'cert_cn' in executable_dict:
            #Add the unique exectuable to a dictionary with certs
            unique_executables[executable_dict['path']] = santa_data_structure(executable_dict['cert_cn'],executable_dict['sha256'],executable_dict['cert_sha256'])
            #Lets add the unique publisher if it doesn't exist already
            if executable_dict['cert_cn'] not in unique_publishers:
                unique_publishers[executable_dict['cert_cn']] = executable_dict['cert_sha256']
        else:
            #This isn't a signed executable, just add the path
            unique_executables[executable_dict['path']] = santa_data_structure(executable_dict['sha256'])
        #print(unique_executables)

Lets print out all the information

print("Unique Executables:") for key, value in unique_executables.items(): print(" Path of the executable: %s" % key) print(" Common name of the certificate that signed the executable: %s" % value.executable_cn) print(" SHA256 hash of the executable: %s" % value.executable_hash) print(" SHA256 hash of the cert that signed the executable: %s\n" % value.cert_hash)

print("Unique Publishers:") for key, value in unique_publishers.items(): print(" Common Name: %s" % key) print(" SHA256 hash of the cert that signed the executable: %s\n" % value)

[dremspider]

It could also be possible to use this tool for forensic history. Run the command and see all the dates and arguments a particular commaND was run. I noticed that it appears that the log file isn't persistent and seems to get wiped out? Maybe after reboot?

[russellhancox]

We don't have anything like this but it certainly is a good idea. I'd prefer to see it added as a command to santactl, using the SNTFileInfo class to grab the details though, rather than looking through the log.

The log file is rotated by ASL when it reaches a certain size. For forensic history we're relying on either/or:

1. Having some system collect the log from the machine regularly, which many organizations running Macs already do.
2. Connecting clients to a server so Santa can upload execution events for analysis and management of the rules.

Though I can definitely see value in evaluating all binaries on a host when Santa is first installed but this would need to be run in the background with as little performance penalty as possible.

[dremspider]

Thank you very much for you time. I have a total of about 2 weeks experience with Mac systems. Most of my experience came from managing security in Linux enterprise systems and figured I would give it a shot. I have been looking to learn more about the OS and how it differs from Linux. I wasn't aware of ASL, but I see now it is very similiar to sylog/logrotate, thanks for the info.

I could see two purposes of a tool like this:

1. The what the heck is being launched on my system approach... This is how I used it. I ran your tool in monitor mode for a little bit and then clicked on everything I could find and let the logs fill up. I then ran this tool and got a list of all the developers/paths and made exceptions to them as I saw fit. Ideal, no but it worked.
2. The maintaining a baseline approach. This is how I have used most enterprise whitelisting tools in the past. We build a system which has all the software our users have and we only use it for the purpose of whitelisting so that we can be sure it won't be compromised. When software needs to be added/updated we deploy to this system first, record the changes and add it to our whitelist. We then push the software/updates.

Perhaps it would be possible to create a "unique" executable table in the SQLite database. When a file is read, the program first checks if it is trusted, if it isn't see if the file has been seen before. If it hasn't it adds an entry to this table. It would then be possible to go in and easily accept or investigate the executables. This would make the monitor mode a lot more useful and ensure that when you do lock it down, you can be confident you got most of the files. Santactl could then be used to pull this information from the table. It would also be possible to warn (maybe add a notification in the notification center) when a hash has changed to turn the monitor mode into a simple file integrity monitor.

Not sure if this approach would be much better, I have never programmed in objective-c so I am not sure how useful I would be though maybe now is a good time to learn :-).

Some things I learned while using the tool... I didn't have a config.plist. I had to create it, no idea why. Once I did, I couldn't modify it. When I would save the file with vi and then save it, it would revert back. I found that I had to stop the service before editing the file with: kextunload /Library/Extensions/santa-driver.kext/ launchctl unload /Library/LaunchDaemons/com.google.santad.plist

Then I could modify it, the start it up again.

One area that I was really confused with was santactl for adding rules. I thought that path would add an exception that would allow the executable that is at that path regardless of the hash or binary at that location and sha256 would allow any executable with that sha hash to execute (i.e. sha256sum). I later figure out that path would allow that executable to run with that specific hash and that sha256 was allowing by a certificate. I would recommend making this more clear in the usage information:

• (NSString *)longHelpText { return (@"Usage: santactl rule {add|remove} [options]\n" @" --whitelist: add to whitelist\n" @" --blacklist: add to blacklist\n" @" --silent-blacklist: add to silent blacklist\n" @" --message {message}: custom message\n" @" --path {path}: Will match on any executable with the hash of the file as specified in the path\n" @" --sha256 {sha256}: Will match on any executable which has been signed by a certificate with the sha256 hash specified\n"); }

Other than that, the tool is pretty easy to use! So far I haven't had any issues.

[russellhancox]

Santa actually keeps a database of blocked or allowed-but-unknown binaries in a second SQLite database called events.db, these events are intended to be uploaded to a central server where you would manage the whitelist. We're still working on the server side and hope to have it open sourced before the end of the year.

As our intention is to use Santa with this server, I haven't spent much time thinking about single user machines or using Santa without a server, though I can certainly see the appeal of this. When I have a bit more time, I'll investigate this further.

Without a config.plist Santa should still run, using some very basic defaults. Did you get an error when you tried?

Some of the keys in the config.plist are protected when santad is running, as you found.

For the rule help text: good suggestion, 0f3a22878848102daa12f4690b1fb7e03ae90233.

[dremspider]

Config.plist, no it worked fine, I was just confused because I was trying to set it to block and the file wasn't there.

The eventsdb would have been useful for me earlier. How do I read this?

95|a8c7932565aaa1cc98337e1a22bc39495af7b4d1b41f4f1e2853677b8d191d06|bplist00?z{X$versionX$objectsY$archiverT$top

eventsdata clearly is what I need, but I am not sure what it all means. How do I pull the file path/hash/certificate from this to be able to create a rule to allow this or is this somehow encrypted so only the server can read this?

61|13a2f6fa0a166324139ac00dfb9cb5bda14b1900e7e9e06c1e981eb5d8b751a2|bplist00?JKX$versionX$objectsY$archiverT$top

If I know this I could easily develop a python script to pull this data and accomplish something similar to what I was doing before with the event logs before. I started looking the code, I have some C background, though haven't touched it in a few years. I though how different could objective C be... Boy was I wrong, brackets everywhere, pluses and minuses in front of functions?/methods? I think... I started looking into it to see if I could do anything.

Some thoughts on how to make this really work well for personal users:

1. Have a setting for enterprise or personal mode.
2. Allow the user to modify the settings/view the events from an application (preference pane?).
3. When the user is prompted for the executable being blocked give them the option to create whitelist rules (with a password of course).

What would be really awesome, though a lot of work is some sort of file reputation service. All users submit their newly seen hashes to a server (scale matters here, so it would be better as a cloud type service), the server on the other end would rate the executable. The rating would be low, if the executable was never seen before (or seen only recently) and would be high if it has seen lots of users. This of course is me dreaming.

I will continue messing with objective c, but I can't promise anything. All my GUI experience is visual studio/c# so I have a bit of a learning curve. I would imagine that I could use your santactl example to see if I could get the GUI added.

[russellhancox]

The events in the database are archived Objective-C objects, you can extract them with:

sqlite3 /var/db/santa/events.db "SELECT writefile(idx || '.plist', eventdata) FROM events;"

And read them with plutil -convert xml1 -o - x.plist.

Unfortunately the format is not intended for easy consumption outside of the app which produced it.

[dremspider]

Very cool, this would have made it a lot easier.

I could begin working on a Python tool to pull this information.

https://docs.python.org/3/library/plistlib.html https://docs.python.org/3/library/plistlib.html

Python 2.X comes with Mac I believe though I had to install Python 3.X. My code currently is 3.X, but it should be easy for me to convert it to 2.X. It appears that the python that is included includes plistlib to de-serialize this information so the user wouldn’t need to install anything extra.

Would you be interested in a tool such as this?

I don’t have the skills to do this in Objective-C at this time.

On Sep 29, 2015, at 3:19 PM, Russell Hancox [email protected] wrote:

plutil -convert xml1 -o - x.plist

[dremspider]

I have been spending a bit of time learning objective-C. I now can at least understand the code a bit better. Objective-C is an interesting mix of C and its own weird concoction. I agree that it would be better for pulling from this events.db database. From my brief looking at SntFileInfo, it appears this is the class which is called for getting information about a file (i.e. who signed it, what is it's hash, etc). I would like to pull the information directly from the database because the file may be long gone by the time you get to the database.

I am going to try to get to eventually playing with it and maybe adding a command to santactl to dump all the information from the database and make it in a more user friendly format. It could be a real, practical way for me to learn objective-c. All my experience has been with C#, so it could be a fun way to learn.

[russellhancox]

Sorry, I'd meant to respond to this sooner but got sidetracked.

The class that is serialized into the database is actually SNTStoredEvent (SNTFileInfo gathers the data as you identified but it's stored by way of SNTStoredEvent which has no logic in it other than the ability to be transported between processes). As santactl isn't meant to access the databases directly, there's an RPC system for asking the daemon to send over events that are in the database (as used in the sync command) and it will handle the deserializing back into objects for you.

Adding this is a command to santactl should be relatively easy. I'd suggest using the status command as a starting template as it's quite short but still requires a daemon connection. You can then look at SNTCommandSyncEventUpload, which shows how to get the events via the daemon (lines 36 and 57).

I'm always happy to help if you get stuck, we have a Google Group called santa-dev that you can sign up and post to.

[pmarkowsky]

Closing this as we've not done it in over 5 years. We may revisit this at some point in the future but I think that needs another feature request.