Allow to separate -x outputs with the match name

[Porkepix]

Currently when using fd -x to execute a command on every match of what we look for with fd, if it's a command where the output and where does it come from matters, we should be able to know which result is associated with this output with a separator between them (ideally with a toggle on/off, probably defaulting to off to keep the default as it currently is).

[dogsc729]

Hi! 👋 I’m currently working on this feature. Here’s the separator format I came up with — I’d love to hear your suggestions or preferences:

---- fd: cat /path/to/file/test1.txt ----
hello
---- fd: cat /path/to/file/test2.txt ----
world

This behavior is controlled by a new flag, --exec-print-separator. The separator will only be printed when this flag is explicitly set.

This is also my first time contributing to an open-source project, and I’m really excited to get involved! 🚀

[Porkepix]

Hi! 👋 I’m currently working on this feature. Here’s the separator format I came up with — I’d love to hear your suggestions or preferences:

---- fd: cat /path/to/file/test1.txt ----
hello
---- fd: cat /path/to/file/test2.txt ----
world

This behavior is controlled by a new flag, --exec-print-separator. The separator will only be printed when this flag is explicitly set.

This is also my first time contributing to an open-source project, and I’m really excited to get involved! 🚀

First of all, thanks for tackling on the issue!

Format is fine to me, even just the file name followed by a semicolon would have been fine to me! I feel like the flag wording could be a little bit confusing as I read it in a way that suggest you could pick what the separator will be. Take this with a grain of salt though: English isn't my native language and it's completely fine to me this way (but @sharkdp's input would be welcome for flags choice and homogeneity).

I just thought some kind of whitelist of commands that would toggle the flag automatically by default (cat, ls, …) would be welcome but have a relevant list would prove to be difficult I think, so maybe not that much a good idea. But maybe get this always on or behind a flag to toggle this off would be useful?

[dogsc729]

@Porkepix Thanks a lot for the feedback! 🙏

I see what you mean about the flag name — something like --exec-enable-separator might make the intent clearer. I also agree that having the option the other way around (separator on by default, with a flag to disable) could work too. Maybe best to hold off until we hear what @sharkdp thinks about naming and defaults for consistency.

[tmccombs]

FWIW, you can accomplish something like this today with:

$ fd -x printf "---- fd: cat %s ----"  {} \; -x cat

That does execute two commands for each input, but given there is a pretty straightforward workaround, I'm not sure it is worth adding a new option, unless there is a compelling reason why that would be better.

p.s. For cat specifically, I've often wished that there was an option I could give to cat to tell it to print the name of the files before outputting them...

[Porkepix]

FWIW, you can accomplish something like this today with:

$ fd -x printf "---- fd: cat %s ----"  {} \; -x cat

That does execute two commands for each input, but given there is a pretty straightforward workaround, I'm not sure it is worth adding a new option, unless there is a compelling reason why that would be better.

p.s. For cat specifically, I've often wished that there was an option I could give to cat to tell it to print the name of the files before outputting them...

Well, in all fairness, I didn't knew you could pile -x up (maybe I didn't read manpage well enough or forgot from back then, or the feature came later on), however note the "If parallelism is enabled, the order commands will be executed in is non-deterministic." in the manpage that could result in a barely usable output if the printf command would come apart from the cat one.

I'd also argue that like half the commands a user would want to run in there result in something unusable if we don't know from which file/directory does it come from: as soon as the user want to see/check things and not act on them, like listing directories contents, showing file content, showing grep results (need to know where the grep comes from!) and so on.

[tmccombs]

could result in a barely usable output if the printf command would come apart from the cat one

If you have multiple commands, and there are multiple threads, we buffer the output to make sure the output from all the commands run for a single file are output together.

[Porkepix]

could result in a barely usable output if the printf command would come apart from the cat one

If you have multiple commands, and there are multiple threads, we buffer the output to make sure the output from all the commands run for a single file are output together.

In this case, what's the point of this warning in the manpage?

[tmccombs]

So in the example command I gave, if you had four files that were found: a, b, c, and d, then the printf for "a" and the cat for "a" would be next to each other, and the output from printf will come before the output from cat. However, there is no deterministic order for what order you will get the output for each file. You could get in the order a,b,c,d for one run, but c,a,d,b the next run. And the processes could run in parallel, even though the output is serialized. Which could be important if your command has side effects, like say if you ran rm -r.

[Porkepix]

So in the example command I gave, if you had four files that were found: a, b, c, and d, then the printf for "a" and the cat for "a" would be next to each other, and the output from printf will come before the output from cat. However, there is no deterministic order for what order you will get the output for each file. You could get in the order a,b,c,d for one run, but c,a,d,b the next run. And the processes could run in parallel, even though the output is serialized. Which could be important if your command has side effects, like say if you ran rm -r.

Mmmh, the whole paragraph looks confusing to me in that case (but that might be because English isn't my native language): the way I read it was the following:

• With only several -x and several commands, each -x argument could be run independently from each other, and same for files, such as [a - command 2], [c - command 1], [a - command 1], [b - command 1], [b - command 2], …
• When restricting to a single thread with --thread=1, then each -x command would be run sequentially for a given file, but file order could still come in a random order in the way you describe.

[tmccombs]

If you have suggestions on how to make it clearer, I'd be happy to hear them.