$MFT parsing

[MariasStory]

Hi team,

Thanks for the cool tool.

I am using iped-4.0.3 on windows.

Maybe I am missing something, but it seems that an $MFT file that is present in a folder given to IPED is not parsed.

It is indexed and I can find the filename that I am looking for. The findings is in a file like "$MFT_39". But, I don't see the the dates and additional information.

To reproduce:

1. Copy an MFT file to a folder
2. Parse the folder with iped.exe
3. Search a file name in the IPED-SearchApp.exe

Expected result: The file with details from $MFT should be present in the table.

Please explain.

[lfcnassif]

Hi @MariasStory,

Not sure if I understood, have you extracted the $MFT using some low level tool (e.g. FTKImager or IPED) to a mounted folder and tried to process that folder?

If yes, it seems ok. The expected behavior is to index that standalone $MFT using the Strings parser, since we don't have a specific "parser" for standalone $MFT files. When processing physical devices or images, the $MFT is decoded under the hood by TSK library together with other needed FS information. So this is done as part of decoding complete devices/images. IPED tool was not designed for your use case, although we could try to implement such a parser, but I don't think it is a common scenario...

[MariasStory]

Hi @lfcnassif, It would be very useful to implement this feature, as it is often used. It is called a triage image. You collect the most important part of the evidence in order to find out the details of what was going on, without collecting full image. In this case, you may miss something, but it is sufficient to answers 90% of the cases.

[MariasStory]

One more thing, I find it useful to have the string indexing of the $MFT, there is some information that is not parsed by another tools. But, it is also important to have the data properly parsed in order to make sense of it.

[lfcnassif]

It is called a triage image. You collect the most important part of the evidence in order to find out the details of what was going on, without collecting full image. In this case, you may miss something, but it is sufficient to answers 90% of the cases.

edited: Have you tried to use the -profile fastmode option? The input can be the physical device, acquired image or a mounted folder, but it basically just does file system decoding and transversal, all other modules are disabled. Usually it takes a few minutes to finish (well the running time increased in about 50% after we upgraded to TSK-4.11 - actually it increased by 5x and I had to do some patches on TSK to disable things we don't use). Then you can browse the file system structure, file categories (based on extension only), view file content, use the gallery, filter results based on file metadata and some other useful features for triage scenarios. Finally you can export relevant files to a custom ZIP file preserving the folder structure and basic file metadata, like MACB dates.

But, it is also important to have the data properly parsed in order to make sense of it.

edited: Do you know some command line tool to parse standalone $MFT files that works both on Windows and Linux and with a compatible license (MIT, BSD, Apache, Mozilla, LGPL) to eventually be integrated? If yes, one possible integration option is configuring it as an external parser: https://github.com/sepinf-inc/IPED/wiki/User-Manual#external-tool-integration

[MariasStory]

Maybe this one: https://github.com/omerbenamram/mft

[lfcnassif]

Maybe this one: https://github.com/omerbenamram/mft

The license is ok, thanks.

Just importing and rendering its CSV output format should be straightforward.

About extracting each record as a different item into the case to populate the table timeline view/chart, it is possible. But, when processing full images (maybe the most common use case), this would duplicate all MACB events, because they are already extracted when performing the FS decoding. So, this could be left as a future improvement. If implemented, I think extracting events from parsed $MFT should be disabled by default. Maybe it is possible to check if the event was already added to the case and add if not found, not sure if matching returned events is easy, since they come from different tools, maybe using the mft record number + sequece number would help, if they are extracted by above tool.

[MariasStory]

Hi @lfcnassif, I shortly checked the mft_dump-v0.6.0.exe, and conclude that:

1. The tools speed is good
2. Levels of details is good, especially in json format
3. I did not try the "--extract-resident-streams" option, but find it very interesting and important, as the evidence may be there

I want to point out that we partially discussed the issue of missing "FN info times from MFT" https://github.com/sepinf-inc/IPED/issues/289

I see two possible solutions:

1. Switch the $MFT parsing to the new tool
2. Combine the old $MFT processing with the output from the new tool and deduplicate (FN times should be implemented)

In any case, the "extract-resident-streams" should be done by the new tool or implemented in the old one.

[MariasStory]

Hi @lfcnassif, thank you again for developing this great tool! I'm still missing the parsing of $MFT in a folder. It would be fantastic to have this feature implemented. In case there are multiple $MFT files in the directory being parsed, it might become confusing as we need to distinguish between "normal" entries and those from found $MFT files with identification of the file that the entry is parsed from. This improvement would significantly enhance the usefulness of IPED for me.

[lfcnassif]

Hi @MariasStory.

Thanks, all contributors deserve your congrats.

Unfortunately the feature requests queue keeps growing, we are not being able to implement all desired features. This use case is not common to my agency, collecting $MFT files and processing them in a stand alone way, if I understood it correctly... Unfortunately , we have to prioritize use cases common to us... There are many ready pull requests waiting for my review I should focus on before. But if you can implement a proposal and send a pull request, I'll be happy to review and test.