WithSecureLabs
MFT Library Fork
As we experienced in #212 there were several issues discovered with the mft library being used. It turns out a lot of the issues we have been experiencing have already been fixed by others. As a result I forked the library over to here with cherry picked commits from https://github.com/omerbenamram/mft/pull/120, https://github.com/omerbenamram/mft/pull/119 and https://github.com/hodf-cye/mft/commit/e5c8b665ff13395df6e8706ab1ee07634f38ae03 as well as bumping up some of the rust dependencies. I haven't bumped them all yet due to some having significant changes that need testing. The intention for this is for either the cloning of the repo I have started or me transferring it over and then we can change it to the WithSecureLabs repos instead of being external.
One issue that will need fixed is when dumping to JSON it outputs as one / instead of // and I believe JSON wants them escaped. CSV is fine however.
Managed to bump up the dependencies to latest which should help with maintainability going forward. I also fixed the benchmark file too that didn't account for changes in the code. When running tests on the library you should run: cargo test -- --skip "readme_sect_library_usage_line_33" as skeptic isn't checking the readme properly (not a deal breaker)
Bitflags was the most troublesome due to them changing the way they serialised the output. I modified this code to use fmt:Display to restore the prior output however one output difference if you use the mft_dump binary the library outputs using the json output format the IndexRootFlags appear to output both SMALL_INDEX and LARGE_INDEX instead of previously just outputting LARGE_INDEX. I am not sure if this is a bug yet but it would be good to figure that one out.
For ease (my side), I have forked mft, applied the two pending PRs and handled the bitrot. I have not applied the timestamp commit but I can add that in. Due to that though I am going to close this as we can just raise PRs for mft against my fork.
@alexkornitzer Sounds good to me. Mine was a dirty hack job anyway just enough for a POC. If any of my commits are useful feel free to steal them/ take bits out for your version.
Do check the formatting differences with migrating to bitflags 2 with the IndexRootFlags and any other weird differences by comparing the outputs of the older library and the newer one as I did observe differences between them which concerned me.
I will be testing this in the coming weeks when I get a chance again myself and will report/ troubleshoot anything odd I notice between versions
Yep, if you could test at some point that would be great. Hence why I have released as beta for now.
The bitflags changes should be fine as I have not replaced the serde with version 2. Instead I have left it using the in library macros that mft was using.
What features do you need in to unblock your other PR?
In terms of unblocking the other PR the issue was the forward backslash differences between OSes so that was the only one stopping it being merged as far as I am aware.
There may be other issues with the MFT library but the “Use consistent path separator on all OSes” commit fixes the main issue holding it up