Notify seems to trigger on `std::fs::copy` but not `std::io::copy` or `std::fs::{read,write}`

[casey]

System details

• OS: MacOS v10.14.6
• Rust version: rustc 1.45.0 (5c1f21c3b 2020-07-13)
• Notify version: v4.0.15 from crates.io
• Filesystem: APFS (Encrypted)

Description

I encountered this issue using cargo-watch, but I think it's a notify issue. Forgive me if that's not the case! I'm using cargo-watch v0.7.5 built from source from crates.io, which uses watchexec v1.14.0, which uses notify v4.0.15.

I have a static site generator, and my watch command was running in a loop. The path update debug message looked like this:

2020-09-04T23:27:52.237-07:00 - DEBUG - Paths updated: [PathOp { path: "/Users/rodarmor/src/blog/in/2xxx/index.png", op: Some((empty)), cookie: None }, …

A bunch of similar paths were also printed. All the files were static assets in the in directory, and they were files that I didn't think I was modifying.

The static assets were all being copied from the in directory to an ignored out directory with the following code:

 std::fs::copy(in_src, out_dst)?;

When I replaced the above with the functionally equivalent, cargo-watch stopped looping:

let data = std::fs::read(in_src)?;
std::fs::write(out_dst, &data)?;

I also tried the following, which is even closer, and which also stopped the looping:

let mut src = std::fs::File::open(src).unwrap();
let mut dst = std::fs::File::create(dst).unwrap();
std::io::copy(&mut src, &mut dst).unwrap();

I dug into std::fs::copy to see if I could find anything suspicious, but I couldn't find anything. For reference, std::fs::copy is defined here.

This is super strange, since I think the above snippets are mostly identical, and none of them seem to modify the assets in in.

I'm trying to get dtrace working to see if I can see any difference in system calls, but in the mean time I thought I'd create this issue and see if it was known.

[casey]

Dtrace has yielded many secrets!

Here is what seem to be the relevant system calls from the version with std::fs::copy, which triggers notify:

stat64("www/blog/2xxx/index.png\0", 0x7FFEE17652B0, 0x0)		 = -1 Err#2
open("in/2xxx/index.png\0", 0x1000000, 0x1B6)		 = 3 0
fstat64(0x3, 0x7FFEE17651F0, 0x0)		 = 0 0
fclonefileat(0x3, 0xFFFFFFFFFFFFFFFE, 0x7FD6A64115C0)		 = 0 0
close_nocancel(0x3)		 = 0 0

It looks like fclonefileat, which creates a copy-on-write copy of a file, is what is ultimately creating a copy of the file. (The last argument to fclonefileat is a path, which is surely www/blog/2xxx/index.png.

And here are the system calls from the version with io::copy, which does not trigger notify:

stat64("www/blog/2xxx/index.png\0", 0x7FFEEB9612B0, 0x0)		 = -1 Err#2
open("in/2xxx/index.png\0", 0x1000000, 0x1B6)		 = 3 0
open("www/blog/2xxx/index.png\0", 0x1000601, 0x1B6)		 = 4 0
close_nocancel(0x4)		 = 0 0
close_nocancel(0x3)		 = 0 0

I couldn't actually identify a system call that was doing the copying, but there were definitely no calls to fclonefileat in the dtrace output.

Does notify perhaps trigger when a file is cloned with fclonefileat?

[anp]

I'm seeing similar behavior with some code that could also be called a static site generator if one were to squint aggressively.

Is this worth discussing upstream as an issue for libstd? I wouldn't expect std::fs::copy to generate filesystem notifications or any observable mutation to the source file. Maybe that's too naive on my part?

[0xpr03]

Is this mac-OS only ? I do not have any mac machine to test things

[casey]

Is this mac-OS only ? I do not have any mac machine to test things

I'm guessing that this is macOS-only, since the fclonefileat system call is only available on macOS.