Add WindowsIndexSearch parsing using ESEDatabaseView

[Karneades]

Description

Add WindowsIndexSearch parsing using ESEDatabaseView. Currently, we collect the Windows Index Search DB through !BasicCollection but lack a module to parse it. One way is to use the ESEDatabaseView which extracts some basic infos from the DB.

I put it into the FS folder because it gives file and folder information.

Checklist:

Please replace every instance of [ ] with [X]

• [x] I have generated a unique GUID for my Target(s)/Module(s)
• [x] I have placed the Target/Module in an appropriate subfolder in Targets or Modules. If one doesn't exist, I have either added it to the Misc folder or created a relevant subfolder with justification
• [x] I have set or updated the version of my Target(s)/Module(s)
• [x] I have verified that KAPE parses the Target successfully via kape.exe, using --tlist/--mlist and corrected any errors
• [x] I have made an attempt to document the artifacts within the Target(s) or Module(s) I am submitting. If documentation doesn't exist, I have placed N/A underneath the Documentation header
• [ ] I have consulted either the Target Guide, Target Template, Compound Target Guide, or Compound Target Template to ensure my Target(s) follow the same format

Thank you for your submission and for contributing to the DFIR community!

[EricZimmerman]

that tool is not ideal at all. look at sum data with it. its all wrong.

if we need a parser for windowsindexsearch, i will write one, like sumecmd.

do we have test data?

[Karneades]

I know that it's not ideal... timestamps and other infos are messed up etc. But at least it gives the file/folder information. The other tool I referenced in the doc section is https://github.com/moaistory/WinSearchDBAnalyzer.

I don't have public test data.

[EricZimmerman]

I'm gonna look at that other tool. Looks like he may have tolled his own ese parser. Would be a good idea to compare his with native windows.

[AndrewRathbun]

that tool is not ideal at all. look at sum data with it. its all wrong.

if we need a parser for windowsindexsearch, i will write one, like sumecmd.

do we have test data?

https://github.com/log2timeline/plaso/blob/main/test_data/Windows.edb

[AndrewRathbun]

@EricZimmerman if you want, I can provide my personal 1GB Windows.ebd file for testing purposes.

[AndrewRathbun]

Windows.zip

Actually, here's the LoneWolf2018 Windows.ebd file.

[EricZimmerman]

I will just write my own parser here I think. That way we control things as far as formatting, export, etc

[AndrewRathbun]

If there's any way Maps can be implemented like your other tools, that would be awesome 👍

[EricZimmerman]

Depends on what the data looks like I guess

[AndrewRathbun]

From what I've seen, it looks pretty similar to SQLite DBs with multiple tables. However, I don't know what we'd use to write queries since it doesn't speak SQL and wasn't designed to accept direct queries. If anything, maybe a Map would look similar to SQLECmd in structure (with IdentifyQuery and IdentifyValue), but with elements of RECmd Batch Files with BinaryInclude, BinaryConvert:FILETIME, and all that type of stuff. And likely it'd just have to be a complete dump of all tables within a DB with the similar naming structure as SQLECmd. So, instead of a SQL query, you basically have various conversion properties for a given column within the db.

Here's a mockup of what I'm talking about:

Description: SRUDB.dat - SRUM Database
Author: Andrew Rathbun
Email: [email protected]
Id: db337b07-2e96-4052-91df-f230dfbd2324
Version: 1.0
CSVPrefix: SRUM
FileName: SRUDB.dat
IdentifyQuery: SELECT count(*) FROM sqlite_master WHERE type='table' AND (name='{DA73FB89-2BEA-4DDC-86B8-6E048C6DA477}' OR name='{FEE4E14F-02A9-4550-B5CE-5FA2DA202E37}LT' OR name='{FEE4E14F-02A9-4550-B5CE-5FA2DA202E37}' OR name='{D10CA2FE-6FCF-4F6D-848E-B2E99266FA89}' OR name='{5C8CF1C7-7257-4F13-B223-970EF5939312}' OR name='{7ACBBAA3-D029-4BE4-9A7A-0885927F1D8F}' OR name='{DD6636C4-8929-4683-974E-22C046A43763}' OR name='{D10CA2FE-6FCF-4F6D-848E-B2E99266FA86}' OR name='{973F5D5C-1D90-4944-BE8E-24B94231A174}' OR name='SruDbCheckpointTable' OR name='SruDbIdMapTable' OR name='MSysLocales' OR name='MSysObjids' OR name='MSysObjectsShadow' OR name='MSysObjects');
IdentifyValue: 15
Queries:
    -
        Name: SRUM SruDbIdMapTable
        Query: |
                Table: SruDbIdMapTable
                Column: EventTimestamp
                    - 
                      BinaryInclude:true
                      BinaryConvert:FILETIME
                Column: Blah
                    - 
                BinaryInclude:true
                BinaryConvert:FILETIME
        BaseFileName: SruDbIdMapTable

# rinse and repeat above for each table within an ESE DB that you want to manipulate/convert the values within a specified column based on logic provided with tool. If nothing is specified here, then expect just a full dump of a DB without any manipulation. Just raw values. 
# Another idea could be ValuePrefix or ValueSuffix, i.e., a column within SRUM DB is Bytes Sent, so ValueSuffix: 'MB' would make it so there's MB appended afterwards, `1234MB`, for example. 

Basically, a full dump of all tables and columns are implied, but the Map maker could impact how some columns are interpreted like with RECmd Batch Files. So the CSV output would look like the following:

SRUM_{DA73FB89-2BEA-4DDC-86B8-6E048C6DA477}.csv
SRUM_{FEE4E14F-02A9-4550-B5CE-5FA2DA202E37}LT.csv
SRUM_{FEE4E14F-02A9-4550-B5CE-5FA2DA202E37}.csv
SRUM_{D10CA2FE-6FCF-4F6D-848E-B2E99266FA89}.csv
SRUM_{5C8CF1C7-7257-4F13-B223-970EF5939312}.csv
SRUM_{7ACBBAA3-D029-4BE4-9A7A-0885927F1D8F}.csv
SRUM_{DD6636C4-8929-4683-974E-22C046A43763}.csv
SRUM_{D10CA2FE-6FCF-4F6D-848E-B2E99266FA86}.csv
SRUM_{973F5D5C-1D90-4944-BE8E-24B94231A174}.csv
SRUM_SruDbCheckpointTable.csv
SRUM_SruDbIdMapTable.csv
SRUM_MSysLocales.csv
SRUM_MSysObjids.csv
SRUM_MSysObjectsShadow.csv
SRUM_MSysObjects.csv

I realize the above example is for the SRUM DB, of which you have a dedicated parser, but that's just the ESE DB I had open already for the purpose of this demonstration. I could see an ESE DB --hunt switch being useful like in SQLECmd.

Anyways, if you need me to do any legwork to run down what some DBs contain, just ping me. Whatever I can do to help 👍

[Beercow]

@MarkBaggett has a tool for this. You could write a plugin for it. https://github.com/MarkBaggett/ese-analyst

[EricZimmerman]

python? ill pass. =)

[randomaccess3]

@AndrewRathbun randomly checked this pull request and @EricZimmerman has mentioned he'll write a parser ;)

[randomaccess3]

@Karneades suggest testing this https://github.com/strozfriedberg/sidr and then writing a module for that. Ill test it out next week during class, and make a module if you dont

[AndrewRathbun]

@Karneades suggest testing this https://github.com/strozfriedberg/sidr and then writing a module for that. Ill test it out next week during class, and make a module if you dont

This is on my list too. Seemed like a pretty good tool. Will help to have a Module in place.

[Karneades]

Thanks for bringing this up @randomaccess3 - eventually I find the time at the weekend. If @AndrewRathbun is on it earlier, then go for it :)

[gs3cl]

I'm not Andrew but here we go a good starting point I think :)

https://github.com/EricZimmerman/KapeFiles/pull/800

[AndrewRathbun]

I'm not Andrew but here we go a good starting point I think :)

https://github.com/EricZimmerman/KapeFiles/pull/800

Yes, great starting point. I think we already have the Target covered though: https://github.com/EricZimmerman/KapeFiles/blob/master/Targets/Windows/WindowsIndexSearch.tkape

Will test the Module tomorrow and make any changes to the PR, if needed.

[Karneades]

Hi, thanks for making the module 👍🏽

Regarding the target, there‘s already one: https://github.com/EricZimmerman/KapeFiles/blob/master/Targets/Windows/WindowsIndexSearch.tkape

I would omit specifing the file name, so we cover Windows 10 and 11. In 11 the file was renamed to Windows.db and uses now the SQLite format.

The tool [SIDR] handles both ESE databases (Windows.edb) and SQLite databases (Windows.db)

I would just improve documenation regarding Windows 11 in the existing target.

Am Sonntag, 14. Mai 2023 um 15:05, gs3cl @.***(mailto:Am Sonntag, 14. Mai 2023 um 15:05, gs3cl < schrieb:

I'm not Andrew but here we go a good starting point I think :)

#800

[gs3cl]

Welcome ! :)

Yes you are right. Can do more testing next week but I think @AndrewRathbun will step in thats fine. I have no Windwos 11 at the moment to test the thing with windows.db.

[AndrewRathbun]

Welcome ! :)

Yes you are right. Can do more testing next week but I think @AndrewRathbun will step in thats fine. I have no Windwos 11 at the moment to test the thing with windows.db.

https://github.com/AndrewRathbun/DFIRArtifactMuseum/tree/main/Windows%2FWindowsSearchDB

I have a Windows 11 sample here but it doesn't appear to be .DB. I'll have to add a sample one. I can do that tomorrow.

[Karneades]

Further reference, see section "Changes in Windows 11": https://www.aon.com/cyber-solutions/aon_cyber_labs/windows-search-index-the-forensic-artifact-youve-been-searching-for/. It seems to be the output of the research around Windows Index Search and the Stroz Friedberg's SIDR tool.

Stroz Friedberg is the renowned DFIR consulting practice of Aon Cyber Solutions, and we love open source!

[Karneades]

I close this as we now have the new module for SIDR.

[oid2000]

Windows 11 now uses SQLite database named Windows.db to store the Search artifact. The structure seems to be the same