duc -new-files-only

[j4jes]

Would duc be able to consider how recent files are when indexing? Someone on my server wrote a lot of data last night, but I don't see any evidence pointing to where it inflated yet. It would be nice if duc had an argument that only indexed files, or displayed files in the index written in the last 24-48 hrs. This would solve many storage abuse mysteries for me, better then du -sh * can.

[zevv]

Quoting Jesse of the North (2018-02-21 16:56:52)

Would duc be able to consider how recent files are when indexing? Someone on my server wrote a lot of data last night, but I don't see any evidence pointing to where it inflated yet. It would be nice if duc had an argument that only indexed files, or displayed files in the index written in the last 24-48 hrs. This would solve many storage abuse mysteries for me, better then du -sh * can.

That seems like a valid use case to me, but I'm not sure about how the implementation should work. The simplest would probably to add an option to 'duc index' to only consider files for indexing with a ctime, mtime or atime newer then a given timestamp (absolute or relative), so that would be something like

duc index --newer-then 3d /

or

duc index --newer-then 2006-08-14 /

Would that work for you?

-- :wq ^X^Cy^K^X^C^C^C^C

[l8gravely]

I think instead it would be more interesting if instead you showed which areas grew the most since the last index, with brighter colors or thicker lines showing the growth areas.

Otherwise when I need to find my biggest growth users in the past twenty four hours, I used to take snapshots of the quota reports from my nfs file servers.

Duc is more for overall usage, but doesn't do well for historical or trending. Finding the most recent files that changed is nice, but still takes time on a filessytem with 30 million files.

Doing tracking like this at the filesystem layer would be ideal, but had a lot of overhead and performance implications...

Sent from my iPhone

On Feb 21, 2018, at 12:07 PM, Ico Doornekamp [email protected] wrote:

Quoting Jesse of the North (2018-02-21 16:56:52)

Would duc be able to consider how recent files are when indexing? Someone on my server wrote a lot of data last night, but I don't see any evidence pointing to where it inflated yet. It would be nice if duc had an argument that only indexed files, or displayed files in the index written in the last 24-48 hrs. This would solve many storage abuse mysteries for me, better then du -sh * can.

That seems like a valid use case to me, but I'm not sure about how the implementation should work. The simplest would probably to add an option to 'duc index' to only consider files for indexing with a ctime, mtime or atime newer then a given timestamp (absolute or relative), so that would be something like

duc index --newer-then 3d /

or

duc index --newer-then 2006-08-14 /

Would that work for you?

-- :wq ^X^Cy^K^X^C^C^C^C — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[skruseman]

I'm experimenting with running duc index and duc ls on a daily basis and storing the ls output in git. Then look at the topmost diffs with git. Did anyone try something like this?