Add `--ctime` (change time) filter option to `fd`

[Inkbottle007]

Dear fd maintainers,

I’d like to suggest an enhancement for fd: the inclusion of a --ctime (change time) filter option. Currently, sorting and filtering by modification time (mtime) is quite common, which is certainly useful in many scenarios. However, there are instances where ctime provides additional value.

For example, after downloading a file like the "C ANSI manual" (http://www1.cs.columbia.edu/~sedwards/papers/sgi1999c.pdf), one might expect to find it at the top of a sorted directory list. But if the mtime hasn’t been altered since the download, it might not appear at the top as expected, even though its ctime has been updated upon download.

I find myself relying on ctime in these scenarios to locate recently downloaded or moved files that do not have a recent mtime. While this is the primary use case that prompts my request, I’m confident others could benefit from a ctime filter as well.

I noticed there has been some previous discussion on this topic (https://github.com/sharkdp/fd/issues/165#issuecomment-1459979793), indicating I’m not alone in valuing this feature.

I believe adding a --ctime option could be a valuable enhancement to fd's functionality.

Thank you for your time and for maintaining such a useful tool.

Best regards, Chris

[dimbtp]

Hello, @Inkbottle007 , I can implement this but I have some questions.

You want to locate recently downloaded or moved files that do not have a recent mtime, but after I search a bit, I find there are two candidates that may work differently: btime(birth time) and ctime(change time).

btime always keep same after you created that file. While ctime changes in many situations such as move, rename, modify, change owner, etc.

For recently downloaded file scenario, if you download several files under folder A, some days later, you rename part of those files, then their ctimes change and you cannot track all these files together which may be related in some way. However, use btime here, you can always know when these files are added to your storage.

For moved files scenario, ctime is all you want.

I'm not sure if I'm overthinking it or if it goes beyond common scenarios, but I think btime is more suitable for your use case and ctime can be changed very easily. Of course, this can be limited with existing other arguments under some scenarios.

[dimbtp]

Hi, @sharkdp , I find this issue has been open for 2 weeks and I don't know whether this feature is necessary or not in your repo.

If I'm allowed to do this, I have questions about what should I do to the original arguments related to mtime, which occupies the name change while its correct name should be modified.

Users are accustomed to using --change*. Would it be inappropriate to suddenly change it to --modified*? Because this tool is widely used in many scripts.

Should I use --ctime-before --ctime-within for ctime? Or I can keep --newer and --older (which may be used more often by current users) for mtime, but then give --changed-before back to ctime?

[Inkbottle007]

btime is the newest addition but atime, ctime and mtime are well established. I myself am only interested in the addition of ctime. Regarding the naming, using --change* for change time, even if it really makes sense, would be a breaking change I suppose. In contrast --ctime-before, --ctime-within would be unambiguous, and allow for "human-friendly time expressions". For consistency, for people using --ctime-before and --change-before in the same script, the introduction of --mtime-before as an alias to --change-before might help make things clearer?

[tmccombs]

Using brime requires support not only from the OS, but from the filesystem. On linux at least, getting btime requires a separate syscall, statx, than the stat we currently use. I'm not saying it can't be done, but it isn't quite as simple as just comparing a different attribute.

[dimbtp]

I just noticed that. btime can be obtained easily from metadata.created(), but ctime seems harder to get on Windows. I am trying to search something useful.

[Inkbottle007]

I just noticed that. btime can be obtained easily from metadata.created(), but ctime seems harder to get on Windows. I am trying to search something useful.

Implementing btime won't help with this issue which is about ctime.

[dimbtp]

On Windows, I find some data structures about ChangeTime(ctime for Windows), but I don't know how to get this value as I don't have much experience with it. Maybe wait for other people's help. :face_exhaling:

On Linux/Unix, it's easy to make it work for ctime. If you work on linux, I can fork one and make a patch.

fd supports multiple platforms, I'm not sure a linux only feature will get merged in release? You may have to download and compile it by yourself.

[tmccombs]

ctime on MetadataExt is available on all Unix variants, which would include all supported OSes except for windows.

I don't think that being unix only would be a blocker to merging it. It wouldn't be the first option that is specific to unix oses.

[dimbtp]

I am trying to write test for --ctime option, however according to https://lists.gnu.org/archive/html/coreutils/2010-08/msg00010.html, ctime _must_ unfakeably track the current time of any action that changes a file's metadata or contents, utimensat syscall can only modify atime and mtime, crate filetime(use this syscall on unix) can't change ctime with mtime.

Here are 2 options to handle this

• use date command to change current system time, file use system time to update their ctime. It's OK to change date in GitHub actions, but in a local test, you need to run with root and it's kind of annoying to change system time.
• use bindfs --ctime-from-mtime FOLDER1 FOLDER2, which will mount FOLDER1 into FOLDER2, all the files and folders under FOLDER2 will change their ctime according to mtime. It's OK to do that in GitHub actions, but in a local test, you may need to grant root privilege for bindfs and umount commands, however, other tests don't need that.

I don't have much experience with this, can you give me some suggestions?

[Inkbottle007]

On Windows, I find some data structures about ChangeTime(ctime for Windows), but I don't know how to get this value as I don't have much experience with it. Maybe wait for other people's help. 😮‍💨

On Linux/Unix, it's easy to make it work for ctime. If you work on linux, I can fork one and make a patch.

fd supports multiple platforms, I'm not sure a linux only feature will get merged in release? You may have to download and compile it by yourself.

I get it. I wasn't aware of that point that Windows doesn't have a "change time". So it seems that the timestamp features of findutils just can't be ported to a platform independent tool. That's it then, can't be done. The issue should just be closed I suppose. Thanks for bringing that point to my attention.

[dimbtp]

the timestamp features of findutils just can't be ported to a platform independent tool

Yes, it's platform dependent at this moment.

I wasn't aware of that point that Windows doesn't have a "change time".

Windows do have corresponding ctime. As code snippet in rust source:

#[repr(C)]
pub struct FILE_BASIC_INFO {
    pub CreationTime: i64, // btime
    pub LastAccessTime: i64, // atime
    pub LastWriteTime: i64, // mtime
    pub ChangeTime: i64, // ctime
    pub FileAttributes: u32,
}

but I don't have much experience with Rust on Windows, I also tried crate windows-sys (Windows API for rust), but I didn't make it. :face_with_spiral_eyes:

This feature could be implemented by someone who is more experienced.

[tavianator]

See https://github.com/rust-lang/rust/issues/112327

[taziden]

I just got bitten by the fact that --changed-(within|before) is actually based on mtime and not ctime. I was using fdfind to detect files created during a malware intrusion and some files were not detected because mtime was faked by the attacker. Being able to search on ctime seems like a must-have.

[Inkbottle007]

It happened to me too. I was naively expecting that --changed-(within|before) was meaning "change time". It is confusing and I don't see how it can be changed.

[tmccombs]

Yeah, at this point I don't think it would be possible to change the behavior of the existing options.

[juliusl]

Saw my issue was tagged here so just dropping a note, I got the ok w/ my PR to add change_time as an unstable feature. Here is the tracking issue -- https://github.com/rust-lang/rust/issues/121478