dust icon indicating copy to clipboard operation
dust copied to clipboard

Add flag to ignore hard links

Open EmNudge opened this issue 3 years ago • 2 comments

Attempting to calculate the size of a directory after using tools which take advantage of hard links to reduce duplication across directories. It's not enough to only count hard links a single time in a run since those hard links all originate elsewhere and the originals won't be caught by dust or du.

Not even du has this feature afaik, so it would be a nice bonus to have from dust.

Alternatively, we can follow the behavior of du in that listing multiple directories allows us to use hard links from previous dirs in ignoring later dirs. When trying to filter out packages from pnpm on macos, for example, we can run dust -sh ./Library/pnpm ./our-folder and we should be removing all the hardlinks from ./our-folder that have been counted already from ./Library/pnpm.

EmNudge avatar Sep 09 '22 04:09 EmNudge

So an option that ignores hard-links? I imagine it should ignore soft-links as well?

Do you think the links should be completely ignored or should they be listed but just as near-empty files ?

bootandy avatar Sep 11 '22 07:09 bootandy

Yeah, an option to ignore all symbolic links would be nice. You can keep them, but list the size of the symbolic link instead of the content it points to, so it would show as a very small file. I guess the option would more accurately be named "don't follow symlinks" rather than ignore them.

EmNudge avatar Sep 11 '22 16:09 EmNudge

I don't think this is possible.

https://unix.stackexchange.com/questions/122333/how-to-tell-which-file-is-original-if-hard-link-is-created

It is not possible to know which is a hard-link and which is the original

bootandy avatar Oct 19 '22 22:10 bootandy

That's unfortunate. Perhaps then just a flag to ignore duplicate files? It would accidentally capture legitimate duplicates, but it would still solve a bit of the problem.

EmNudge avatar Nov 22 '22 01:11 EmNudge

it already ignores duplicate files. If 2 files have the same inode it will only count them once.

bootandy avatar Jan 04 '23 21:01 bootandy