pauldreik
./testcases/largefilesupport.sh fails for 1.7
After update to 1.7, ./testcases/largefilesupport.sh fails on 32bit x86 and arm:
[ 28s] FAIL: testcases/largefilesupport.sh
[ 28s] ===================================
[ 28s]
[ 28s] largefilesupport.sh: checking for rdfind ... OK.
[ 28s] checking for valgrind ...not used.
[ 28s] checking for mktemp ... OK.
[ 28s] ./testcases/largefilesupport.sh debug: temp dir is /tmp/rdfindtestcases.d.aTxXm5NLtX3v
[ 28s] 0+0 records in
[ 28s] 0+0 records out
[ 28s] 0 bytes copied, 3.427e-05 s, 0.0 kB/s
[ 28s] 0+0 records in
[ 28s] 0+0 records out
[ 28s] 0 bytes copied, 2.556e-05 s, 0.0 kB/s
[ 28s] Now scanning "subdir", found 0 files.
[ 28s] Now have 0 files in total.
[ 28s] Removed 0 files due to nonunique device and inode.
[ 28s] Total size is 0 bytes or 0 B
[ 28s] Removed 0 files due to unique sizes from list. 0 files left.
[ 28s] Now eliminating candidates based on first bytes: removed 0 files from list. 0 files left.
[ 28s] Now eliminating candidates based on last bytes: removed 0 files from list. 0 files left.
[ 28s] Now eliminating candidates based on sha1 checksum: removed 0 files from list. 0 files left.
[ 28s] It seems like you have 0 files that are not unique
[ 28s] Totally, 0 B can be reduced.
[ 28s] Now making results file results.txt
[ 28s] ./testcases/largefilesupport.sh debug: rdfind ran ok.
[ 28s] FAIL testcases/largefilesupport.sh (exit status: 1)
In particular, this fails:
grep -q "^Total size is $((filesize * 2)) bytes" rdfind.out
See this sh -xx output with ls -l added to the test:
-rw-r--r-- 1 abuild abuild 2147483648 Feb 10 08:14 sparse-file1
-rw-r--r-- 1 abuild abuild 2147483648 Feb 10 08:14 sparse-file2
+ mkdir subdir
+ mv sparse-file1 sparse-file2 subdir
+ /home/abuild/rpmbuild/BUILD/rdfind-1.7.0-build/rdfind-1.7.0/rdfind subdir
+ tee rdfind.out
Now scanning "subdir", found 0 files.
Now have 0 files in total.
Removed 0 files due to nonunique device and inode.
Total size is 0 bytes or 0 B
Removed 0 files due to unique sizes from list. 0 files left.
Now eliminating candidates based on first bytes: removed 0 files from list. 0 files left.
Now eliminating candidates based on last bytes: removed 0 files from list. 0 files left.
Now eliminating candidates based on sha1 checksum: removed 0 files from list. 0 files left.
It seems like you have 0 files that are not unique
Totally, 0 B can be reduced.
Now making results file results.txt
+ dbgecho 'rdfind ran ok.'
+ echo './testcases/largefilesupport.sh debug: ' 'rdfind ran ok.'
./testcases/largefilesupport.sh debug: rdfind ran ok.
+ grep -q '^Total size is 4294967296 bytes' rdfind.out
+ cleanup
+ cd /
+ rm -rf /tmp/rdfindtestcases.d.W7cKwXuwSvmg
With 1.6:
+ tee rdfind.out
Now scanning "subdir", found 2 files.
Now have 2 files in total.
Removed 0 files due to nonunique device and inode.
Total size is 4294967296 bytes or 4 GiB
Removed 0 files due to unique sizes from list. 2 files left.
Now eliminating candidates based on first bytes: removed 2 files from list. 0 files left.
Now eliminating candidates based on last bytes: removed 0 files from list. 0 files left.
Now eliminating candidates based on sha1 checksum: removed 0 files from list. 0 files left.
It seems like you have 0 files that are not unique
Totally, 0 B can be reduced.
Now making results file results.txt
+ dbgecho 'rdfind ran ok.'
+ echo './testcases/largefilesupport.sh debug: ' 'rdfind ran ok.'
./testcases/largefilesupport.sh debug: rdfind ran ok.
+ grep -q '^Total size is 4294967296 bytes' rdfind.out
+ grep -q '^It seems like you have 0 files that are not unique$' rdfind.out
+ cleanup
+ cd /
+ rm -rf /tmp/rdfindtestcases.d.rtAE9CxBTcZV
This looks suspicious:
Now scanning "subdir", found 0 files.
vs.
Now scanning "subdir", found 2 files.
OK, it's autoconf, it looks (you used 2.69 for 1.6 and 2.72 for 1.7):
checking for option to enable large file support... support not detected
vs.
checking for _FILE_OFFSET_BITS value needed for large files... 64
So lstat() fails with EOVERFLOW in Dirlist::walk().
Note the empty space between for option in 1.7 output. CC is unset, apparently at that point. gcc (apart from g++) is installed.
Time to switch to https://github.com/mesonbuild/meson?
autoconf is funny, because it sets:
ac_compile='$CXX -c $CXXFLAGS $CPPFLAGS conftest.$ac_ext >&5'
which is used in ac_fn_cxx_try_compile. But when checking for the -D_FILE_OFFSET_BITS=64 option, it does:
for ac_opt in "none needed" "-D_FILE_OFFSET_BITS=64" "-D_LARGE_FILES=1" "-n32"; do
...
ac_save_CC="$CC"
CC="$ac_save_CC $ac_opt"
but that has apparently no effect as ac_compile uses CXX, not CC.
When I s/CC/CXX/ there, it all works and config.h now writes:
#define _FILE_OFFSET_BITS 64
Please get rid of this autotools crap.
This was reported some time ago at: https://savannah.gnu.org/support/?110983
openSUSE has the fix, so if I rerun autoconf, it works too.
Thanks for digging in to this!
This solves the mystery in https://github.com/pauldreik/rdfind/issues/184
I don't like autotools either. There is unoffical cmake support since a week or two.
Do I understand you correctly that is the generated configure that is the problem, so if I generate it on an older system ( using automake 2.69 ) and make a new .tar.gz, everything is fine?
I don't like autotools either. There is unoffical cmake support since a week or two.
Make it official then ;).
Do I understand you correctly that is the generated configure that is the problem, so if I generate it on an older system ( using automake 2.69 ) and make a new .tar.gz, everything is fine?
Exactly. Or use a fixed system. BTW openSUSE has this patch: https://build.opensuse.org/projects/devel:tools:building/packages/autoconf/files/Port-AC_SYS_LARGEFILE-to-C.patch?expand=1