bulk_extractor
bulk_extractor copied to clipboard
Published
20 hours ago •
simsong
simsong
Build fails on aarch64 and armv7 with LTO
When LTO is enabled, build fails on aarch64 and armv7. Log:
[ 97s] g++ -std=c++17 -g -mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -pthread -Wall -MD -Wpointer-arith -Wshadow -Wwrite-strings -Wcast-align -Wredundant-decls -Wdisabled-optimization -Wfloat-equal -Wmultichar -Wmissing-noreturn -Woverloaded-virtual -Wsign-promo -funit-at-a-time -Weffc++ -fPIC -O3 -flto=auto -L/usr/local/lib -o test_be be20_api/abstract_image_reader.o be20_api/atomic_unicode_histogram.o be20_api/feature_recorder.o be20_api/feature_recorder_file.o be20_api/feature_recorder_set.o be20_api/feature_recorder_sql.o be20_api/histogram_def.o be20_api/path_printer.o be20_api/pcap_fake.o be20_api/pos0.o be20_api/regex_vector.o be20_api/sbuf.o be20_api/sbuf_stream.o be20_api/scan_sha1_test.o be20_api/scanner_config.o be20_api/scanner_params.o be20_api/scanner_set.o be20_api/threadpool.o be20_api/unicode_escape.o be20_api/utils.o be20_api/word_and_context_list.o base64_forensic.o bulk_extractor.o image_process.o notify_thread.o phase1.o sbuf_decompress.o bulk_extractor_scanners.o scan_aes.o scan_base64.o scan_ccns2.o scan_elf.o scan_evtx.o scan_exif.o exif_reader.o exif_entry.o scan_exiv2.o scan_facebook.o scan_find.o scan_gzip.o scan_hiberfile.o pyxpress.o scan_httplogs.o scan_json.o scan_kml.o scan_msxml.o scan_net.o scan_ntfsindx.o scan_ntfslogfile.o scan_ntfsmft.o scan_ntfsusn.o scan_outlook.o scan_pdf.o scan_rar.o scan_sqlite.o scan_utmp.o scan_vcard.o scan_windirs.o scan_winlnk.o scan_winpe.o scan_winprefetch.o scan_wordlist.o scan_xor.o scan_zip.o pcap_writer.o scan_base16.o scan_accts.o scan_email.o scan_gps.o rar/arccmt.o rar/archive.o rar/arcread.o rar/cmddata.o rar/coder.o rar/crc.o rar/encname.o rar/errhnd.o rar/extinfo.o rar/extract.o rar/file.o rar/filefn.o rar/filestr.o rar/getbits.o rar/global.o rar/match.o rar/model.o rar/suballoc.o rar/options.o rar/pathfn.o rar/rar.o rar/rarpch.o rar/rarvmtbl.o rar/rawread.o rar/rdwrfn.o rar/resource.o rar/rs.o rar/savepos.o rar/smallfn.o rar/strfn.o rar/strlist.o rar/system.o rar/timefn.o rar/unicode.o rar/unpack.o test_be1.o test_be2.o test_be3.o -lstdc++ -lexpat -lz -ldl -lcrypto -lexpat
[ 97s] <stdout>:752:8: warning: type 'struct yyguts_t' violates the C++ One Definition Rule [-Wodr]
[ 97s] <stdout>:5689:8: note: a different type is defined in another translation unit
[ 97s] <stdout>:756:19: note: the first difference of corresponding definitions is field 'yyextra_r'
[ 97s] <stdout>:5693:19: note: a field of same name but different type is defined in another translation unit
[ 97s] scan_base16.flex:19:7: note: type name 'base16_scanner' should match type name 'accts_scanner'
[ 97s] 19 | class base16_scanner : public sbuf_scanner {
[ 97s] | ^
[ 97s] scan_accts.flex:20:7: note: the incompatible type is defined here
[ 97s] 20 | class accts_scanner : public sbuf_scanner {
[ 97s] | ^
[ 97s] <stdout>:752:8: warning: type 'struct yyguts_t' violates the C++ One Definition Rule [-Wodr]
[ 97s] <stdout>:5689:8: note: a different type is defined in another translation unit
[ 97s] <stdout>:756:19: note: the first difference of corresponding definitions is field 'yyextra_r'
[ 97s] <stdout>:5693:19: note: a field of same name but different type is defined in another translation unit
[ 97s] scan_base16.flex:19:7: note: type name 'base16_scanner' should match type name 'accts_scanner'
[ 97s] 19 | class base16_scanner : public sbuf_scanner {
[ 97s] | ^
[ 97s] scan_accts.flex:20:7: note: the incompatible type is defined here
[ 97s] 20 | class accts_scanner : public sbuf_scanner {
[ 97s] | ^
[ 117s] In member function '__ct ',
[ 117s] inlined from 'add_cpuid' at be20_api/dfxml_cpp/src/dfxml_writer.h:232:23,
[ 117s] inlined from 'add_DFXML_execution_environment' at be20_api/dfxml_cpp/src/dfxml_writer.h:516:18,
[ 117s] inlined from 'add_DFXML_creator' at be20_api/dfxml_cpp/src/dfxml_writer.h:151:40,
[ 117s] inlined from 'dfxml_write_create' at phase1.cpp:246:34:
[ 117s] be20_api/dfxml_cpp/src/cpuid.h:33:9: error: impossible constraint in 'asm'
[ 117s] 33 | asm volatile
[ 117s] | ^
[ 117s] In member function '__ct ',
[ 117s] inlined from 'vendor' at be20_api/dfxml_cpp/src/cpuid.h:51:23,
[ 117s] inlined from 'add_cpuid' at be20_api/dfxml_cpp/src/dfxml_writer.h:238:47,
[ 117s] inlined from 'add_DFXML_execution_environment' at be20_api/dfxml_cpp/src/dfxml_writer.h:516:18,
[ 117s] inlined from 'add_DFXML_creator' at be20_api/dfxml_cpp/src/dfxml_writer.h:151:40,
[ 117s] inlined from 'dfxml_write_create' at phase1.cpp:246:34:
[ 117s] be20_api/dfxml_cpp/src/cpuid.h:33:9: error: impossible constraint in 'asm'
[ 117s] 33 | asm volatile
[ 117s] | ^
[ 117s] In member function '__ct ',
[ 117s] inlined from 'add_cpuid' at be20_api/dfxml_cpp/src/dfxml_writer.h:249:32,
[ 117s] inlined from 'add_DFXML_execution_environment' at be20_api/dfxml_cpp/src/dfxml_writer.h:516:18,
[ 117s] inlined from 'add_DFXML_creator' at be20_api/dfxml_cpp/src/dfxml_writer.h:151:40,
[ 117s] inlined from 'dfxml_write_create' at phase1.cpp:246:34:
[ 117s] be20_api/dfxml_cpp/src/cpuid.h:33:9: error: impossible constraint in 'asm'
[ 117s] 33 | asm volatile
[ 117s] | ^
[ 118s] make[3]: *** [/tmp/cc0BkQQS.mk:26: /tmp/ccirSUV4.ltrans8.ltrans.o] Error 1
[ 118s] make[3]: *** Waiting for unfinished jobs....
[ 120s] lto-wrapper: fatal error: make returned 2 exit status
[ 120s] compilation terminated.
[ 120s] /usr/lib64/gcc/aarch64-suse-linux/12/../../../../aarch64-suse-linux/bin/ld: error: lto-wrapper failed
[ 120s] collect2: error: ld returned 1 exit status
[ 120s] make[2]: *** [Makefile:1229: bulk_extractor] Error 1
[ 120s] make[2]: *** Waiting for unfinished jobs....
[ 138s] In member function '__ct ',
[ 138s] inlined from 'add_cpuid' at be20_api/dfxml_cpp/src/dfxml_writer.h:232:23,
[ 138s] inlined from 'add_DFXML_execution_environment' at be20_api/dfxml_cpp/src/dfxml_writer.h:516:18,
[ 138s] inlined from 'add_DFXML_creator' at be20_api/dfxml_cpp/src/dfxml_writer.h:151:40,
[ 138s] inlined from 'dfxml_write_create' at phase1.cpp:246:34:
[ 138s] be20_api/dfxml_cpp/src/cpuid.h:33:9: error: impossible constraint in 'asm'
[ 138s] 33 | asm volatile
[ 138s] | ^
[ 138s] In member function '__ct ',
[ 138s] inlined from 'vendor' at be20_api/dfxml_cpp/src/cpuid.h:51:23,
[ 138s] inlined from 'add_cpuid' at be20_api/dfxml_cpp/src/dfxml_writer.h:238:47,
[ 138s] inlined from 'add_DFXML_execution_environment' at be20_api/dfxml_cpp/src/dfxml_writer.h:516:18,
[ 138s] inlined from 'add_DFXML_creator' at be20_api/dfxml_cpp/src/dfxml_writer.h:151:40,
[ 138s] inlined from 'dfxml_write_create' at phase1.cpp:246:34:
[ 138s] be20_api/dfxml_cpp/src/cpuid.h:33:9: error: impossible constraint in 'asm'
[ 138s] 33 | asm volatile
[ 138s] | ^
[ 138s] In member function '__ct ',
[ 138s] inlined from 'add_cpuid' at be20_api/dfxml_cpp/src/dfxml_writer.h:249:32,
[ 138s] inlined from 'add_DFXML_execution_environment' at be20_api/dfxml_cpp/src/dfxml_writer.h:516:18,
[ 138s] inlined from 'add_DFXML_creator' at be20_api/dfxml_cpp/src/dfxml_writer.h:151:40,
[ 138s] inlined from 'dfxml_write_create' at phase1.cpp:246:34:
[ 138s] be20_api/dfxml_cpp/src/cpuid.h:33:9: error: impossible constraint in 'asm'
[ 138s] 33 | asm volatile
[ 138s] | ^
[ 139s] make[3]: *** [/tmp/ccjmatNq.mk:29: /tmp/ccxn3mtH.ltrans9.ltrans.o] Error 1
[ 139s] make[3]: *** Waiting for unfinished jobs....
[ 142s] lto-wrapper: fatal error: make returned 2 exit status
[ 142s] compilation terminated.
[ 142s] /usr/lib64/gcc/aarch64-suse-linux/12/../../../../aarch64-suse-linux/bin/ld: error: lto-wrapper failed
[ 142s] collect2: error: ld returned 1 exit status
[ 142s] make[2]: *** [Makefile:1233: test_be] Error 1
[ 142s] make[2]: Leaving directory '/home/abuild/rpmbuild/BUILD/bulk_extractor-2.0.0/src'
[ 142s] make[1]: *** [Makefile:416: all-recursive] Error 1
[ 142s] make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/bulk_extractor-2.0.0'
[ 142s] make: *** [Makefile:357: all] Error 2
When I disable LTO, the build finishes without error.
Thanks for sending this out. 1 - What is the operating system? 2 - What does LTO stand for and how do you enable it? Please edit the issue to provide the full "configure" command. 3 - Looks like you are also being tracked up by the dfxml stuff under ARM.
If you can get me access to a system running your OS and CPU, I can try to fix this. Or you can give it a try!
On 2): LTO is "link-time optimization." It's enabled with "-flto". GCC docs here: https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html
On Mon, Jun 13, 2022 at 8:47 AM Simson L. Garfinkel < @.***> wrote:
Thanks for sending this out. 1 - What is the operating system? 2 - What does LTO stand for and how do you enable it? Please edit the issue to provide the full "configure" command. 3 - Looks like you are also being tracked up by the dfxml stuff under ARM.
If you can get me access to a system running your OS and CPU, I can try to fix this. Or you can give it a try!
— Reply to this email directly, view it on GitHub https://github.com/simsong/bulk_extractor/issues/360#issuecomment-1153871616, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABKVVLZUWBUMXCHXPYOTF3VO4UWRANCNFSM5YT5PZNQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>
- openSUSE Tumbleweed
- LTO is link-time optimization. See comment above and/or https://en.opensuse.org/openSUSE:LTO
Here is the log for configure:
[ 64s] + CONFIG_SHELL=/usr/bin/bash
[ 64s] + export CONFIG_SHELL
[ 64s] + CFLAGS='-mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g'
[ 64s] + export CFLAGS
[ 64s] + CXXFLAGS='-mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g'
[ 64s] + export CXXFLAGS
[ 64s] + FFLAGS='-mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g '
[ 64s] + export FFLAGS
[ 64s] + FCFLAGS='-mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g '
[ 64s] + export FCFLAGS
[ 64s] + LDFLAGS=-flto=auto
[ 64s] + export LDFLAGS
[ 64s] + ./configure --host=aarch64-suse-linux-gnu --build=aarch64-suse-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info
[ 66s] configure: Compiling under aarch64-suse-linux-gnu.
[ 66s] configure: checking /usr/local/include
[ 66s] configure: *** ADDING /usr/local/include to CPPFLAGS ***
[ 66s] configure: *** ADDING /usr/local/lib to LDFLAGS ***
[ 66s] configure: checking /opt/local/include
[ 66s] configure: checking /sw/include
[ 66s] configure: checking /usr/local/ssl/include
[ 66s] configure: CPPFLAGS = -I/usr/local/include
[ 66s] configure: LDFLAGS = -flto=auto -L/usr/local/lib
[ 66s] configure: Including dfxml_configure.m4 from dfxml
[ 66s] configure: Note: checks for libewf.h should be in the caller, so they can be disabled
[ 66s] checking for aarch64-suse-linux-gnu-gcc... no
[ 66s] checking for gcc... gcc
[ 69s] checking whether the C compiler works... yes
[ 69s] checking for C compiler default output file name... a.out
[ 70s] checking for suffix of executables...
[ 72s] checking whether we are cross compiling... no
[ 73s] checking for suffix of object files... o
[ 73s] checking whether the compiler supports GNU C... yes
[ 74s] checking whether gcc accepts -g... yes
[ 74s] checking for gcc option to enable C11 features... none needed
[ 75s] checking whether gcc understands -c and -o together... yes
[ 76s] checking for stdio.h... yes
[ 76s] checking for stdlib.h... yes
[ 77s] checking for string.h... yes
[ 78s] checking for inttypes.h... yes
[ 78s] checking for stdint.h... yes
[ 79s] checking for strings.h... yes
[ 80s] checking for sys/stat.h... yes
[ 80s] checking for sys/types.h... yes
[ 81s] checking for unistd.h... yes
[ 82s] checking for expat.h... yes
[ 83s] checking for sys/resource.h... yes
[ 83s] checking for sys/utsname.h... yes
[ 83s] checking for unistd.h... (cached) yes
[ 84s] checking for winsock2.h... no
[ 85s] checking for boost/version.hpp... no
[ 85s] checking for pwd.h... yes
[ 86s] checking for uuid/uuid.h... no
[ 87s] checking for gmtime_r... yes
[ 89s] checking for getuid... yes
[ 90s] checking for gethostname... yes
[ 92s] checking for getpwuid... yes
[ 93s] checking for getrusage... yes
[ 94s] checking for vasprintf... yes
[ 94s] checking for expat.h... (cached) yes
[ 96s] checking for XML_ParserCreate in -lexpat... yes
[ 96s] checking for git... no
[ 96s] configure: WARNING: git not found
[ 97s] checking for CommonCrypto/CommonDigest.h... no
[ 97s] checking for gcrypt.h... no
[ 98s] checking for gpg_strerror in -lgpg-error... no
[ 98s] checking for gcry_md_open in -lgcrypt... no
[ 99s] checking for openssl/aes.h... yes
[ 99s] checking for openssl/bio.h... yes
[ 100s] checking for openssl/evp.h... yes
[ 101s] checking for openssl/hmac.h... yes
[ 101s] checking for openssl/md5.h... yes
[ 102s] checking for openssl/pem.h... yes
[ 103s] checking for openssl/rand.h... yes
[ 104s] checking for openssl/rsa.h... yes
[ 104s] checking for openssl/sha.h... yes
[ 104s] checking for openssl/pem.h... (cached) yes
[ 106s] checking for openssl/x509.h... yes
[ 107s] checking for EVP_get_digestbyname in -lcrypto... yes
[ 109s] checking for SSL_library_init in -lssl... no
[ 111s] checking for gcry_md_open... no
[ 112s] checking for CC_MD2_Init... no
[ 114s] checking for EVP_get_digestbyname... yes
[ 114s] configure: Including be20_configure.m4 from be20_api
[ 114s] checking whether C++ compiler understands ... Adding -pthread to CXXFLAGS
[ 115s] checking whether byte ordering is bigendian... no
[ 115s] checking for dlfcn.h... yes
[ 116s] checking for fcntl.h... yes
[ 116s] checking for limits.h... yes
[ 117s] checking for limits/limits.h... no
[ 117s] checking for linux/if_ether.h... yes
[ 118s] checking for net/ethernet.h... yes
[ 118s] checking for netinet/if_ether.h... yes
[ 119s] checking for netinet/in.h... yes
[ 119s] checking for pcap.h... no
[ 120s] checking for pcap/pcap.h... no
[ 120s] checking for sqlite3.h... no
[ 121s] checking for sys/cdefs.h... yes
[ 121s] checking for sys/mman.h... yes
[ 121s] checking for sys/stat.h... (cached) yes
[ 122s] checking for sys/time.h... yes
[ 122s] checking for sys/types.h... (cached) yes
[ 122s] checking for sys/vmmeter.h... no
[ 122s] checking for unistd.h... (cached) yes
[ 123s] checking for windows.h... no
[ 123s] checking for windows.h... (cached) no
[ 123s] checking for windowsx.h... no
[ 123s] checking for winsock2.h... (cached) no
[ 124s] checking for wpcap/pcap.h... no
[ 124s] checking for mach/mach.h... no
[ 125s] checking for mach-o/dyld.h... no
[ 125s] checking for gmtime_r... (cached) yes
[ 126s] checking for ishexnumber... no
[ 128s] checking for isxdigit... yes
[ 129s] checking for localtime_r... yes
[ 129s] checking for unistd.h... no
[ 131s] checking for mmap... yes
[ 132s] checking for err... yes
[ 134s] checking for errx... yes
[ 135s] checking for warn... yes
[ 137s] checking for warnx... yes
[ 138s] checking for pread64... yes
[ 139s] checking for pread... yes
[ 141s] checking for strptime... yes
[ 142s] checking for _lseeki64... no
[ 144s] checking for task_info... no
[ 145s] checking for utimes... yes
[ 146s] checking for host_statistics64... no
[ 147s] checking for sqlite3_libversion in -lsqlite3... no
[ 148s] checking for sqlite3_create_function_v2... no
[ 149s] checking for sysctlbyname... no
[ 150s] checking for a BSD-compatible install... /usr/bin/install -c
[ 150s] checking whether build environment is sane... yes
[ 150s] checking for a race-free mkdir -p... /usr/bin/mkdir -p
[ 150s] checking for gawk... gawk
[ 150s] checking whether make sets $(MAKE)... yes
[ 150s] checking whether make supports the include directive... yes (GNU style)
[ 151s] checking whether make supports nested variables... yes
[ 151s] checking dependency style of gcc... none
[ 151s] checking whether to enable maintainer-specific portions of Makefiles... no
[ 151s] checking for aarch64-suse-linux-gnu-gcc... gcc
[ 151s] checking whether the compiler supports GNU C... (cached) yes
[ 151s] checking whether gcc accepts -g... (cached) yes
[ 151s] checking for gcc option to enable C11 features... (cached) none needed
[ 151s] checking whether gcc understands -c and -o together... (cached) yes
[ 151s] checking for aarch64-suse-linux-gnu-g++... no
[ 151s] checking for aarch64-suse-linux-gnu-c++... no
[ 151s] checking for aarch64-suse-linux-gnu-gpp... no
[ 151s] checking for aarch64-suse-linux-gnu-aCC... no
[ 151s] checking for aarch64-suse-linux-gnu-CC... no
[ 151s] checking for aarch64-suse-linux-gnu-cxx... no
[ 151s] checking for aarch64-suse-linux-gnu-cc++... no
[ 151s] checking for aarch64-suse-linux-gnu-cl.exe... no
[ 151s] checking for aarch64-suse-linux-gnu-FCC... no
[ 151s] checking for aarch64-suse-linux-gnu-KCC... no
[ 151s] checking for aarch64-suse-linux-gnu-RCC... no
[ 151s] checking for aarch64-suse-linux-gnu-xlC_r... no
[ 151s] checking for aarch64-suse-linux-gnu-xlC... no
[ 151s] checking for aarch64-suse-linux-gnu-clang++... no
[ 151s] checking for g++... g++
[ 152s] checking whether the compiler supports GNU C++... yes
[ 152s] checking whether g++ accepts -g... yes
[ 153s] checking for g++ option to enable C++11 features... none needed
[ 153s] checking dependency style of g++... none
[ 153s] checking for flex... flex
[ 153s] checking for lex output file root... lex.yy
[ 156s] checking for lex library... none needed
[ 157s] checking whether yytext is a pointer... yes
[ 158s] checking whether g++ supports C++17 features with -std=c++17... yes
[ 164s] checking for filesystem... yes
[ 166s] checking for string... yes
[ 166s] configure: C++ Warnings to test: -Wall -MD -Wpointer-arith -Wshadow -Wwrite-strings -Wcast-align -Wredundant-decls -Wdisabled-optimization -Wfloat-equal -Wmultichar -Wmissing-noreturn -Woverloaded-virtual -Wsign-promo -funit-at-a-time -Weffc++
[ 166s] checking whether g++ understands -Wall... yes
[ 167s] checking whether g++ understands -MD... yes
[ 167s] checking whether g++ understands -Wpointer-arith... yes
[ 168s] checking whether g++ understands -Wshadow... yes
[ 168s] checking whether g++ understands -Wwrite-strings... yes
[ 168s] checking whether g++ understands -Wcast-align... yes
[ 169s] checking whether g++ understands -Wredundant-decls... yes
[ 169s] checking whether g++ understands -Wdisabled-optimization... yes
[ 170s] checking whether g++ understands -Wfloat-equal... yes
[ 170s] checking whether g++ understands -Wmultichar... yes
[ 170s] checking whether g++ understands -Wmissing-noreturn... yes
[ 171s] checking whether g++ understands -Woverloaded-virtual... yes
[ 171s] checking whether g++ understands -Wsign-promo... yes
[ 172s] checking whether g++ understands -funit-at-a-time... yes
[ 172s] checking whether g++ understands -Weffc++... yes
[ 172s] configure: flex supports -R
[ 173s] checking for int64_t... yes
[ 173s] checking for special C compiler options needed for large files... no
[ 174s] checking for _FILE_OFFSET_BITS value needed for large files... no
[ 176s] checking POSIX termios... yes
[ 176s] checking for dlfcn.h... (cached) yes
[ 176s] checking for fcntl.h... (cached) yes
[ 176s] checking for inttypes.h... (cached) yes
[ 177s] checking for libgen.h... yes
[ 177s] checking for limits.h... (cached) yes
[ 177s] checking for mmap.h... no
[ 177s] checking for pwd.h... (cached) yes
[ 178s] checking for signal.h... yes
[ 178s] checking for stdint.h... (cached) yes
[ 178s] checking for sys/cdefs.h... (cached) yes
[ 179s] checking for curses.h... no
[ 179s] checking for sys/disk.h... no
[ 180s] checking for sys/fcntl.h... yes
[ 181s] checking for sys/ioctl.h... yes
[ 181s] checking for sys/mman.h... (cached) yes
[ 181s] checking for sys/mmap.h... no
[ 182s] checking for sys/mount.h... yes
[ 183s] checking for sys/param.h... yes
[ 183s] checking for sys/socket.h... yes
[ 183s] checking for sys/stat.h... (cached) yes
[ 183s] checking for sys/types.h... (cached) yes
[ 183s] checking for sys/time.h... (cached) yes
[ 183s] checking for sys/resource.h... (cached) yes
[ 184s] checking for sys/sysctl.h... no
[ 184s] checking for sys/vmmeter.h... (cached) no
[ 185s] checking for term.h... no
[ 185s] checking for time.h... yes
[ 185s] checking for unistd.h... (cached) yes
[ 185s] checking for windows.h... (cached) no
[ 186s] checking for CoreServices/CoreServices.h... no
[ 186s] checking for mach-o/dyld.h... (cached) no
[ 186s] checking for getuid... (cached) yes
[ 186s] checking for getpwuid... (cached) yes
[ 186s] checking for gethostname... (cached) yes
[ 186s] checking for getrusage... (cached) yes
[ 186s] checking for gmtime_r... (cached) yes
[ 188s] checking for getprogname... no
[ 188s] checking for isxdigit... (cached) yes
[ 188s] checking for ishexnumber... (cached) no
[ 189s] checking for le64toh... no
[ 189s] checking for localtime_r... (cached) yes
[ 189s] checking for _lseeki64... (cached) no
[ 191s] checking for inet_ntop... yes
[ 193s] checking for ioctl... yes
[ 195s] checking for isatty... yes
[ 195s] checking for pread64... (cached) yes
[ 195s] checking for pread... (cached) yes
[ 197s] checking for printf... yes
[ 197s] checking for mmap... (cached) yes
[ 198s] checking for munmap... yes
[ 200s] checking for MD5... yes
[ 202s] checking for mkstemp... yes
[ 204s] checking for mktemp... yes
[ 206s] checking for sleep... yes
[ 208s] checking for SleepEx... no
[ 208s] checking for strptime... (cached) yes
[ 210s] checking for usleep... yes
[ 210s] checking for vasprintf... (cached) yes
[ 212s] checking for _NSGetExecutablePath... no
[ 213s] checking for CreateProcess... no
[ 215s] checking for LoadLibrary... no
[ 217s] checking for IncrementAtomic... no
[ 220s] checking for InterlockedIncrement... no
[ 220s] checking for tgetstr in -ltermcap... no
[ 222s] checking for dlopen in -ldl... yes
[ 225s] checking for dlopen... yes
[ 227s] checking for dlopen_preflight... no
[ 227s] checking for arpa/inet.h... yes
[ 227s] checking for netinet/in.h... (cached) yes
[ 228s] checking for wsipx.h... no
[ 228s] checking for netinet/ip.h... yes
[ 229s] checking for struct winsize.ws_col... yes
[ 230s] checking for struct sockaddr_in.sin_len... no
[ 232s] checking for uncompress in -lz... yes
[ 232s] checking for expat.h... (cached) yes
[ 232s] checking for XML_ParserCreate in -lexpat... (cached) yes
[ 234s] checking for main in -lstdc++... yes
[ 234s] configure: libewf is yes
[ 235s] checking for libewf.h... no
[ 235s] configure: WARNING: libewf.h not found; no E01 support
[ 238s] checking for libewf_handle_get_utf8_header_value_notes... no
[ 240s] checking for libewf_error_backtrace_fprint... no
[ 242s] checking for libewf_handle_close... no
[ 245s] checking for libewf_handle_free... no
[ 247s] checking for libewf_handle_read_buffer_at_offset... no
[ 250s] checking for libewf_handle_read_random... no
[ 250s] configure: libewf is now no
[ 252s] configure: adding -O3 to CFLAGS and CXXFLAGS
[ 252s] configure: *************************************
[ 252s] configure: *************************************
[ 252s] configure: PACKAGE_NAME: BULK_EXTRACTOR
[ 252s] configure: PACKAGE_VERSION: 2.0.0
[ 252s] configure: CC: gcc
[ 252s] configure: CXX: g++ -std=c++17
[ 252s] configure: CPPFLAGS: -I/usr/local/include -DUTC_OFFSET=+0000
[ 252s] configure: CFLAGS: -g -mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -fPIC -O3
[ 252s] configure: CXXFLAGS: -g -mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -pthread -Wall -MD -Wpointer-arith -Wshadow -Wwrite-strings -Wcast-align -Wredundant-decls -Wdisabled-optimization -Wfloat-equal -Wmultichar -Wmissing-noreturn -Woverloaded-virtual -Wsign-promo -funit-at-a-time -Weffc++ -fPIC -O3
[ 252s] configure: LIBS: -lstdc++ -lexpat -lz -ldl -lcrypto -lexpat
[ 252s] configure: LDFLAGS: -flto=auto -L/usr/local/lib
[ 252s] configure: WARNING: libewf not installed. Please install libewf-devel for E01 support. See the etc/ directory for details.
[ 253s] checking that generated files are newer than configure... done
[ 253s] configure: creating ./config.status
[ 255s] config.status: creating Makefile
[ 255s] config.status: creating doc/Makefile
[ 255s] config.status: creating doc/latex_manuals/Makefile
[ 256s] config.status: creating src/Makefile
[ 256s] config.status: creating src/tests/Makefile
[ 256s] config.status: creating man/Makefile
[ 257s] config.status: creating python/Makefile
[ 257s] config.status: creating specfiles/Makefile
[ 257s] config.status: creating specfiles/bulk_extractor.spec.m4
[ 258s] config.status: creating tests/Makefile
[ 258s] config.status: creating config.h
[ 258s] config.status: executing depfiles commands
I cannot provide remote access to an Arm machine, but I can try some patches, if any.
Hi. I need the configure command also. Are you running this in AWS? Can you set up a machine for me to log into?
The configure command is provided above. https://github.com/simsong/bulk_extractor/issues/360#issuecomment-1153915841 This is run on the openSUSE build system (OBS) which uses internal Arm machines, without remote access. I have no Arm machine with remote access, sorry. You can use a Raspberry Pi 3 or 4, or any Single Board Computer with an Arm SoC. Not sure what is the current support of github regarding CI/CD on Arm.
Ah, you are correct. I missed it.
I'll see what I can do about finding an OpenSUSE VM in Amazon and bringing up everything.
This will likely get done faster if you can provide a VM for me AWS where I can do the work. Such VMs are available in the Free Tier. I don't know how long it would take for me to get oepnSUSE running. That's why I asked if you could set up the machine for me. Every minute that I work trying to get the VM operational is a minute I'm not working on solving the problem.
I just tried to repro on my M1 Mac by setting -flto=auto on CFLAGS, CXXFLAGS, and LDFLAGS prior to running configure. With clang/llvm, it does not seem to be a problem. I'll install gcc and try that.
I just tried to repro on my M1 Mac by setting
-flto=autoonCFLAGS,CXXFLAGS, andLDFLAGSprior to running configure. With clang/llvm, it does not seem to be a problem. I'll install gcc and try that.
I forgot to mention that this is with GCC12.
Can you try GCC12.1 and let me know how it goes? There are many reports that GCC12 was not fully baked.
Can you try GCC12.1 and let me know how it goes? There are many reports that GCC12 was not fully baked.
This is already version 12.1.0+git27
Hi. I have never downloaded or installed SUSE before. Can you point me at the images that I need to use for installation? I'm on this web page: https://www.suse.com/download/sles/
Thanks.
Okay. I updated DFXML and now I get warnings but no errors:
g++ -std=c++17 -g -flto -pthread -Wall -MD -Wpointer-arith -Wshadow -Wwrite-strings -Wcast-align -Wredundant-decls -Wdisabled-optimization -Wfloat-equal -Wmultichar -Wmissing-noreturn -Woverloaded-virtual -Wsign-promo -funit-at-a-time -Weffc++ -fPIC -O3 -L/usr/local/lib -o bulk_extractor be20_api/abstract_image_reader.o be20_api/atomic_unicode_histogram.o be20_api/feature_recorder.o be20_api/feature_recorder_file.o be20_api/feature_recorder_set.o be20_api/feature_recorder_sql.o be20_api/histogram_def.o be20_api/path_printer.o be20_api/pcap_fake.o be20_api/pos0.o be20_api/regex_vector.o be20_api/sbuf.o be20_api/sbuf_stream.o be20_api/scan_sha1_test.o be20_api/scanner_config.o be20_api/scanner_params.o be20_api/scanner_set.o be20_api/threadpool.o be20_api/unicode_escape.o be20_api/utils.o be20_api/word_and_context_list.o base64_forensic.o bulk_extractor.o image_process.o notify_thread.o phase1.o sbuf_decompress.o bulk_extractor_scanners.o scan_aes.o scan_base64.o scan_ccns2.o scan_elf.o scan_evtx.o scan_exif.o exif_reader.o exif_entry.o scan_exiv2.o scan_facebook.o scan_find.o scan_gzip.o scan_hiberfile.o pyxpress.o scan_httplogs.o scan_json.o scan_kml.o scan_msxml.o scan_net.o scan_ntfsindx.o scan_ntfslogfile.o scan_ntfsmft.o scan_ntfsusn.o scan_outlook.o scan_pdf.o scan_rar.o scan_sqlite.o scan_utmp.o scan_vcard.o scan_windirs.o scan_winlnk.o scan_winpe.o scan_winprefetch.o scan_wordlist.o scan_xor.o scan_zip.o pcap_writer.o scan_base16.o scan_accts.o scan_email.o scan_gps.o rar/arccmt.o rar/archive.o rar/arcread.o rar/cmddata.o rar/coder.o rar/crc.o rar/encname.o rar/errhnd.o rar/extinfo.o rar/extract.o rar/file.o rar/filefn.o rar/filestr.o rar/getbits.o rar/global.o rar/match.o rar/model.o rar/suballoc.o rar/options.o rar/pathfn.o rar/rar.o rar/rarpch.o rar/rarvmtbl.o rar/rawread.o rar/rdwrfn.o rar/resource.o rar/rs.o rar/savepos.o rar/smallfn.o rar/strfn.o rar/strlist.o rar/system.o rar/timefn.o rar/unicode.o rar/unpack.o main.o -lewf -lstdc++ -lexpat -lz -ldl -lcrypto -lexpat -lsqlite3
<stdout>:752:8: warning: type ‘struct yyguts_t’ violates the C++ One Definition Rule [-Wodr]
<stdout>:5689:8: note: a different type is defined in another translation unit
<stdout>:756:19: note: the first difference of corresponding definitions is field ‘yyextra_r’
<stdout>:5693:19: note: a field of same name but different type is defined in another translation unit
scan_base16.flex:19:7: note: type name ‘base16_scanner’ should match type name ‘accts_scanner’
19 | class base16_scanner : public sbuf_scanner {
| ^
scan_accts.flex:20:7: note: the incompatible type is defined here
20 | class accts_scanner : public sbuf_scanner {
| ^
lto-wrapper: warning: using serial compilation of 25 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
g++ -std=c++17 -g -flto -pthread -Wall -MD -Wpointer-arith -Wshadow -Wwrite-strings -Wcast-align -Wredundant-decls -Wdisabled-optimization -Wfloat-equal -Wmultichar -Wmissing-noreturn -Woverloaded-virtual -Wsign-promo -funit-at-a-time -Weffc++ -fPIC -O3 -L/usr/local/lib -o test_be be20_api/abstract_image_reader.o be20_api/atomic_unicode_histogram.o be20_api/feature_recorder.o be20_api/feature_recorder_file.o be20_api/feature_recorder_set.o be20_api/feature_recorder_sql.o be20_api/histogram_def.o be20_api/path_printer.o be20_api/pcap_fake.o be20_api/pos0.o be20_api/regex_vector.o be20_api/sbuf.o be20_api/sbuf_stream.o be20_api/scan_sha1_test.o be20_api/scanner_config.o be20_api/scanner_params.o be20_api/scanner_set.o be20_api/threadpool.o be20_api/unicode_escape.o be20_api/utils.o be20_api/word_and_context_list.o base64_forensic.o bulk_extractor.o image_process.o notify_thread.o phase1.o sbuf_decompress.o bulk_extractor_scanners.o scan_aes.o scan_base64.o scan_ccns2.o scan_elf.o scan_evtx.o scan_exif.o exif_reader.o exif_entry.o scan_exiv2.o scan_facebook.o scan_find.o scan_gzip.o scan_hiberfile.o pyxpress.o scan_httplogs.o scan_json.o scan_kml.o scan_msxml.o scan_net.o scan_ntfsindx.o scan_ntfslogfile.o scan_ntfsmft.o scan_ntfsusn.o scan_outlook.o scan_pdf.o scan_rar.o scan_sqlite.o scan_utmp.o scan_vcard.o scan_windirs.o scan_winlnk.o scan_winpe.o scan_winprefetch.o scan_wordlist.o scan_xor.o scan_zip.o pcap_writer.o scan_base16.o scan_accts.o scan_email.o scan_gps.o rar/arccmt.o rar/archive.o rar/arcread.o rar/cmddata.o rar/coder.o rar/crc.o rar/encname.o rar/errhnd.o rar/extinfo.o rar/extract.o rar/file.o rar/filefn.o rar/filestr.o rar/getbits.o rar/global.o rar/match.o rar/model.o rar/suballoc.o rar/options.o rar/pathfn.o rar/rar.o rar/rarpch.o rar/rarvmtbl.o rar/rawread.o rar/rdwrfn.o rar/resource.o rar/rs.o rar/savepos.o rar/smallfn.o rar/strfn.o rar/strlist.o rar/system.o rar/timefn.o rar/unicode.o rar/unpack.o test_be1.o test_be2.o test_be3.o -lewf -lstdc++ -lexpat -lz -ldl -lcrypto -lexpat -lsqlite3
<stdout>:752:8: warning: type ‘struct yyguts_t’ violates the C++ One Definition Rule [-Wodr]
<stdout>:5689:8: note: a different type is defined in another translation unit
<stdout>:756:19: note: the first difference of corresponding definitions is field ‘yyextra_r’
<stdout>:5693:19: note: a field of same name but different type is defined in another translation unit
scan_base16.flex:19:7: note: type name ‘base16_scanner’ should match type name ‘accts_scanner’
19 | class base16_scanner : public sbuf_scanner {
| ^
scan_accts.flex:20:7: note: the incompatible type is defined here
20 | class accts_scanner : public sbuf_scanner {
| ^
lto-wrapper: warning: using serial compilation of 25 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
lto-wrapper: warning: using serial compilation of 38 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information```
This is on:
[fedora@ip-172-30-4-181 bulk_extractor]$ cat /etc/system-release Fedora release 36 (Thirty Six) [fedora@ip-172-30-4-181 bulk_extractor]$ g++ --version g++ (GCC) 12.1.1 20220507 (Red Hat 12.1.1-1) Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[fedora@ip-172-30-4-181 bulk_extractor]$
However, these are only warnings. It compiles correctly on aarch64 with LTO:
```make[2]: Leaving directory '/home/fedora/bulk_extractor/src'
Making all in src/tests
make[2]: Entering directory '/home/fedora/bulk_extractor/src/tests'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/fedora/bulk_extractor/src/tests'
Making all in python
make[2]: Entering directory '/home/fedora/bulk_extractor/python'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/fedora/bulk_extractor/python'
Making all in specfiles
make[2]: Entering directory '/home/fedora/bulk_extractor/specfiles'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/fedora/bulk_extractor/specfiles'
Making all in tests
make[2]: Entering directory '/home/fedora/bulk_extractor/tests'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/fedora/bulk_extractor/tests'
make[2]: Entering directory '/home/fedora/bulk_extractor'
make[2]: Nothing to be done for 'all-am'.
make[2]: Leaving directory '/home/fedora/bulk_extractor'
make[1]: Leaving directory '/home/fedora/bulk_extractor'
The unit tests run:
[fedora@ip-172-30-4-181 src]$ ./test_be
$ bulk_extractor
$ bulk_extractor -h
$ bulk_extractor -H
$ bulk_extractor -0q -o /tmp/be_tmpc5df50e713d1a28c /home/fedora/bulk_extractor/src/tests/nps-2010-emails.100k.raw
$ bulk_extractor -0q -o /tmp/be_tmpc5df50e713d1a28c /home/fedora/bulk_extractor/src/tests/nps-2010-emails.100k.raw
$ bulk_extractor -0q -f simsong -o /tmp/be_tmp4054f326528623f7 /home/fedora/bulk_extractor/src/tests/pdf_words2.pdf
outdir: "/tmp/be_tmp4054f326528623f7"
DEBUG_5G not set; skipping 5gb-flatfile test
$ bulk_extractor -Eemail --notify_async -1q -o /tmp/be_tmpcd217b49ddf763dd /tmp/30mb-segmented.000
$ bulk_extractor -p 0:64/h /home/fedora/bulk_extractor/src/tests/test_base64json.txt
$ bulk_extractor --notify_async -1qo /tmp/be_tmpa72d78c8a7d6e8dd /home/fedora/bulk_extractor/src/tests/CFReDS001.E01
opening /home/fedora/bulk_extractor/src/tests/CFReDS001.E01
elapsed time: 14s
$ bulk_extractor --notify_async -1qo /tmp/be_tmpcea22490117157da /home/fedora/bulk_extractor/src/tests/email_test.E01
opening /home/fedora/bulk_extractor/src/tests/email_test.E01
opening /home/fedora/bulk_extractor/src/tests/email_test.E02
opening /home/fedora/bulk_extractor/src/tests/email_test.E03
opening /home/fedora/bulk_extractor/src/tests/email_test.E04
## image_fname: ram_2pages.bin outdir: "/tmp/be_tmp1bfee34ada288b89"
## image_fname: test_base16json.txt outdir: "/tmp/be_tmp8a59635a5eb26762"
## output in "/tmp/be_tmp4c306e35a2207202" for accts
## image_fname: hello_elf outdir: "/tmp/be_tmp20da51e777f201bb"
## image_fname: test_hello.gz outdir: "/tmp/be_tmp2c1e63503b8b4548"
## image_fname: test_json.txt outdir: "/tmp/be_tmp5e3606d4f56715fb"
## image_fname: KML_Samples.kml outdir: "/tmp/be_tmp7e3b7a7b0a8191e6"
## image_fname: jpegs.rar outdir: "/tmp/be_tmpdee3f7c67908abcf"
## image_fname: ntlm1.pcap outdir: "/tmp/be_tmp188a0898c3e3890"
## image_fname: ntlm2.pcap outdir: "/tmp/be_tmp48443e9384f7c0df"
## image_fname: ntlm3.pcap outdir: "/tmp/be_tmpa033bc6c2331a179"
## image_fname: ntlm3.pcap outdir: "/tmp/be_tmpcfcc8d979389dd50"
## image_fname: ntlm3.pcap outdir: "/tmp/be_tmp60d9a96e99901288"
## image_fname: ntlm80.pcap outdir: "/tmp/be_tmp1853f49ba2918e27"
## image_fname: domexusers-2435863310-2435928846.raw outdir: "/tmp/be_tmp253753d6f31bd130"
## image_fname: hello_win64_exe outdir: "/tmp/be_tmp8b09ad77652156a"
$ bulk_extractor --notify_async -Ro /tmp/be_tmp8eaa015a1a152215 /home/fedora/bulk_extractor/src/tests
## output in "/tmp/be_tmp93b25a34512f2ce" for email
## output in "/tmp/be_tmpc2f8a284361d8169" for email
## output in "/tmp/be_tmpfd6db20e1f397b92" for pdf
## output in "/tmp/be_tmp18eeef7b41f7f5f4" for email
## output in "/tmp/be_tmp4010ba5f1404bbfe" for exif
## output in "/tmp/be_tmpf370873500a2c805" for exif
## output in "/tmp/be_tmp5b7003328b355692" for exif
## output in "/tmp/be_tmpaaf08f5fa65930db" for json
## output in "/tmp/be_tmpc9286f0f9e2f3f87" for vcard_carved
## output in "/tmp/be_tmp424ae734de8cdacc" for wordlist
## output in "/tmp/be_tmpca6babc140d911a6" for winprefetch
## output in "/tmp/be_tmp509c36eda4248d92" for zip
=======================================================================================================================
All tests passed (100359 assertions in 53 test cases)
[fedora@ip-172-30-4-181 src]$
Can you give https://github.com/simsong/bulk_extractor/commit/d0505fb0fbc4cee08cbab1ed0d8f2802ead4b0f1 a try?
For example, do I want the desktop or the enterprise? Thanks.
Better to install openSUSE Tumbleweed, where the issue is: https://get.opensuse.org/tumbleweed/
Thanks for the recommendation. Before I go to the trouble, can you give d0505fb a try?
I'm closing this because it is not clear to me that LTO is something that is required. If it is, please re-open.