symlink loops in Windows7 ntfs file system hang md5deep

[vanepp]

First let me say thank you for a wonderful program that I have used for years! Environment:

Target file system (to be scanned):

Windows 7 Pro on an Intel machine (this particular one is a new install from the CD when I discovered that some things I had done on my prod box were causing some of the issues I see :-)).

Machine running md5deep (the same Intel machine with an alternate boot disk):

The System Rescue CD (a gentoo linux system that boots to ram as a live CD)

https://www.system-rescue-cd.org

version 4.8.1 (a week or so old) but with one rev back of md5deep 4.3 instead of 4.4 (although I don't think that matters for this as the change log doesn't mention anything that looks likely to fix it).

Boot this up and mount the Windows c: drive (using the ntfs-3g linux ntfs code under the covers):

mount /dev/sda3 /mnt/windows

now two runs of md5deep:

md5deep -r -of /mnt/windows

to avoid device files and mostly links (which appears to be where the problem is). This works as expected and I get lots of md5 values :-) and it completes in a while.

md5deep -r -ofle /mnt/windows

adding in links and pe files (not that I know if I have any pe files :-)) works for a while and then hangs. I expect (but don't have a development environment with gdb on the rescueCD to tell for sure) from looping directories. You need to control-c this to get out, as far as I know it will never complete. Tonite I thought that ls must have this same issue so I tried

ls -lLaR /mnt/windows/* | grep ^l

(to look at what it was doing with links) which had the happy accident of also showing me the error messages which previously were lost in the mass of ls output):

ls: /mnt/windows/Documents and Settings/All Users/Application Data: not listing already-listed directory ls: /mnt/windows/Documents and Settings/Default/AppData/Local/Application Data: not listing already-listed directory ls: /mnt/windows/Documents and Settings/Default/Local Settings/Application Data: not listing already-listed directory ls: /mnt/windows/Documents and Settings/Default User/AppData/Local/Application Data: not listing already-listed directory ... and finally completes. ls (gnu ls in this case) appears to be able to detect symlink loops and deal with them as the ls doesn't hang. Swiping that code for md5deep should fix this problem I expect (although I have no idea how easy or hard that would be, although I expect easier than doing it from scratch!). As well it is my impression (perhaps incorrect) that md5deep is following symlinks. In the past (although I don't have an example) I think md5deep has complained about not being able to access a file from a link to another disk. It seems to me that from a forensic point of view what we should actually be interested in is the value of the symlink not what it points to. In this case if I have symlinks pointing to other file systems, what I care about is the path to the link destination is the same as it was when the hash was taken not the file on the remote disk (which may not be accessable) so changes to this disk are what will be flagged. This issue may have been in a past version and already fixed though.

Peter Van Epp