incompatibility between --dedupe and --partial-only

[reacocard]

What jdupes version are you using?

jdupes 1.20.2 (2021-11-02) 64-bit

Where did you get your copy of the jdupes binary?

[ ] Official binary from jdupes releases [X] OS distribution repository (Linux, Homebrew, etc. - put distro's name/version in text block below) [ ] I personally compiled it from source code (paste your build commands in the text block below) [ ] Other (please explain below)

Debian stable, bullseye-backports repo, package version 1.20.2-1~bpo11+1

Have you searched in Issues?

[X] I have searched for my problem in Issues and I am confident that this is not a duplicate of a previous reported issue or feature request.

I have done my due diligence

[X] I am not suggesting changing the hash algorithm, storing files in a database, comparing file data blocks differently, suggesting that file sizes should be used before comparison, or other common suggestions that indicate I haven't read the documentation, code, or examined the issue tracker entries to discover that all of these things have already been implemented or won't make a difference if they were implemented. I have done my due diligence before asking for help or making a suggestion.

Issue or feature request details

I don't want to spend the IO time fully hashing all the files when the dedupe ioctl is going to bytecheck them anyway, so I tried to use --partial-only with --dedupe, but this results in no action being taken other than printing a cryptic message about --quick, even though I did not specify --quick.

$ jdupes --dedupe --partial-only --partial-only --recurse DIR

BIG FAT WARNING: -T/--partial-only is EXTREMELY DANGEROUS! Read the manual!

--partial-only overrides --quick and is even more dangerous (read the manual!)

$

I believe this is because the --dedupe flag parsing always sets F_QUICKCOMPARE here: https://github.com/jbruchon/jdupes/blob/c92370a919b5bb62960ad965e1e0d2d263ac0084/jdupes.c#L2161

--quick makes sense as a default for --dedupe as hashing the whole file is the only way to ensure the entire file is dedupeable, but for my usecase i would be fine with chancing partial dedupes in order to speed up operations, so I would like --partial-only and --dedupe to work together.

[jbruchon]

Sorry I missed this when you sent it originally. If you'll send a pull request that makes this function without breaing anything, I'll gladly merge it.

[jbruchon]

I've fixed this. I know you've waited a long time. Sorry about that. I've also made -T meaner for normies.

user@host:~/projects/github/jdupes $ ./jdupes -BTTT testdir/
Saying -T three or more times? You're a wizard. No reminders for you.
Scanning: 0 files, 1 dirs (in 1 specified)
  [SRC] testdir/atinydupe0
  -XX-> testdir/Stilltinydupe1
error: Operation not supported (-95)
  -XX-> testdir/tinydupe2
error: Operation not supported (-95)
  -XX-> testdir/Tinydupe3
error: Operation not supported (-95)
  -XX-> testdir/tinydupe4
error: Operation not supported (-95)

  [SRC] testdir/with spaces a
  -XX-> testdir/with spaces b
error: Operation not supported (-95)

  [SRC] testdir/notsotinydupe1
  -XX-> testdir/notsotinydupe2
error: Operation not supported (-95)

  [SRC] testdir/larger_file_1
  -XX-> testdir/larger_file_2
error: Operation not supported (-95)
  -XX-> testdir/larger_file_4
error: Operation not supported (-95)

  [SRC] testdir/.hidden_two
  -XX-> testdir/twice_one
error: Operation not supported (-95)
  -XX-> testdir/two
error: Operation not supported (-95)

Deduplication done (5 files processed)
user@host:~/projects/github/jdupes $ ./jdupes -v
jdupes 1.23.0 (2023-05-08) 64-bit, linked to libjodycode 2.0.1 (2023-05-12)
Compile-time feature flags: dedupe debug loud
Copyright (C) 2015-2023 by Jody Bruchon and contributors

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

If you find this software useful, please consider financially supporting
its development through the author's home page: https://www.jodybruchon.com/
Report bugs and get the latest releases: https://github.com/jbruchon/jdupes