Run tests in CI with address sanitizer (and maybe other sanitizers)
See e.g. godotengine/godot/platform/linuxbsd/detect.py for some compiler/linker flags like -fsanitize=address.
Can confirm that compiling with -fsanitize=undefined,shift,shift-exponent,integer-divide-by-zero,unreachable,vla-bound,null,return,signed-integer-overflow,bounds,float-divide-by-zero,float-cast-overflow,nonnull-attribute,returns-nonnull-attribute,bool,enum,vptr,pointer-overflow,builtin and running tests makes for very interesting output.
Perhaps we can add a ./configure option for this so that one does not have to set CFLAGS manually.
I just came here to report that flint does not pass undefined behavior sanitizer with clang-20. But I see that this is already on your radar. ~~Compiling with CFLAGS="-fsanitize=address" only allows the test suite to pass though. So that's good!~~
EDIT: well, I just noticed that there were some memory leaks in the logs:
gr_generic_poly_factor
0 success, 0 domain, 0 unable 0.19 (PASS)
gr_generic_pow...
=================================================================
==785898==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 960 byte(s) in 6 object(s) allocated from:
#0 0x603ae313c5c4 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x7890629a9b49 in flint_malloc /build/flint-3.3.1-asan/src/generic_files/memory_manager.c:80:18
#2 0x789062ca1628 in fq_zech_ctx_init_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:85:10
#3 0x789062ddac07 in _nmod_mpoly_gcd_algo_small /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:1876:9
#4 0x789062de51df in _nmod_mpoly_gcd_algo /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2246:16
#5 0x789062de51df in nmod_mpoly_gcd /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2272:12
Direct leak of 528 byte(s) in 6 object(s) allocated from:
#0 0x603ae313c5c4 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x7890629a9b49 in flint_malloc /build/flint-3.3.1-asan/src/generic_files/memory_manager.c:80:18
#2 0x789062ca14b5 in fq_zech_ctx_init_fq_nmod_ctx /build/flint-3.3.1-asan/src/fq_zech/ctx.c:251:10
#3 0x789062ca14b5 in _fq_zech_ctx_init_conway_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:64:5
#4 0x789062ca1628 in fq_zech_ctx_init_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:85:10
#5 0x789062ddac07 in _nmod_mpoly_gcd_algo_small /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:1876:9
#6 0x789062de51df in _nmod_mpoly_gcd_algo /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2246:16
#7 0x789062de51df in nmod_mpoly_gcd /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2272:12
Indirect leak of 204 byte(s) in 18 object(s) allocated from:
#0 0x603ae313c5c4 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x7890629a9b49 in flint_malloc /build/flint-3.3.1-asan/src/generic_files/memory_manager.c:80:18
#2 0x789062ca1484 in _fq_zech_ctx_init_conway_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:61:5
#3 0x789062ca1628 in fq_zech_ctx_init_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:85:10
#4 0x789062ddac07 in _nmod_mpoly_gcd_algo_small /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:1876:9
#5 0x789062de51df in _nmod_mpoly_gcd_algo /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2246:16
#6 0x789062de51df in nmod_mpoly_gcd /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2272:12
Indirect leak of 96 byte(s) in 6 object(s) allocated from:
#0 0x603ae313c5c4 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x7890629a9b49 in flint_malloc /build/flint-3.3.1-asan/src/generic_files/memory_manager.c:80:18
#2 0x789062c67358 in fq_nmod_ctx_init_modulus /build/flint-3.3.1-asan/src/fq_nmod/ctx_init_modulus.c:74:5
#3 0x789062ca1484 in _fq_zech_ctx_init_conway_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:61:5
#4 0x789062ca1628 in fq_zech_ctx_init_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:85:10
#5 0x789062ddac07 in _nmod_mpoly_gcd_algo_small /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:1876:9
#6 0x789062de51df in _nmod_mpoly_gcd_algo /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2246:16
#7 0x789062de51df in nmod_mpoly_gcd /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2272:12
Indirect leak of 96 byte(s) in 6 object(s) allocated from:
#0 0x603ae313c5c4 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x7890629a9ba1 in flint_realloc /build/flint-3.3.1-asan/src/generic_files/memory_manager.c:108:16
#2 0x789062ca1484 in _fq_zech_ctx_init_conway_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:61:5
#3 0x789062ca1628 in fq_zech_ctx_init_ui /build/flint-3.3.1-asan/src/fq_zech/ctx.c:85:10
#4 0x789062ddac07 in _nmod_mpoly_gcd_algo_small /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:1876:9
#5 0x789062de51df in _nmod_mpoly_gcd_algo /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2246:16
#6 0x789062de51df in nmod_mpoly_gcd /build/flint-3.3.1-asan/src/nmod_mpoly/gcd.c:2272:12
SUMMARY: AddressSanitizer: 1884 byte(s) leaked in 42 allocation(s).
gr_generic_pow 0.00 (PASS)
fq_zech_is_square 0.04 (PASS)
fq_zech_mul...
make: *** [Makefile:776: build/fmpz_mod_mpoly_factor/test/main_TEST_RUN] Error 1
fq_zech_mul 0.01 (PASS)
The reason I'm investigating this is that I'm seeing some leaks in SymEngine, and it was pointing towards flint as the origin of these leaks.
Did you check with the current main? That particular memory leak may have been fixed in fdb9380, unless there is another, similar bug.
I reran the build and tests on main (d81bf03). Fewer leaks, but still one failing test it seems:
padic_poly_pow...
fmpz_mod_mpoly_q_mul_fmpq 0.01 (PASS)
fmpz_mod_mpoly_q_mul_fmpz...
fmpz_poly_factor_zassenhaus_subset 0.06 (PASS)
=================================================================
==781229==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 70496 byte(s) in 4 object(s) allocated from:
#0 0x5e637d7b3654 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x76e7b61abd69 in flint_malloc /build/flint-d81bf03-asan/src/generic_files/memory_manager.c:80:18
Direct leak of 192 byte(s) in 3 object(s) allocated from:
#0 0x5e637d7b3654 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x76e7b61abd69 in flint_malloc /build/flint-d81bf03-asan/src/generic_files/memory_manager.c:80:18
#2 0x5e637d7f8d2d in main /build/flint-d81bf03-asan/src/fq_default_poly_factor/test/main.c:27:1
Direct leak of 168 byte(s) in 3 object(s) allocated from:
#0 0x5e637d7b3654 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x76e7b61abdc1 in flint_realloc /build/flint-d81bf03-asan/src/generic_files/memory_manager.c:108:16
#2 0x5e637d7f8d2d in main /build/flint-d81bf03-asan/src/fq_default_poly_factor/test/main.c:27:1
Indirect leak of 2304 byte(s) in 18 object(s) allocated from:
#0 0x5e637d7b3654 in malloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:67:3
#1 0x76e7b61abd69 in flint_malloc /build/flint-d81bf03-asan/src/generic_files/memory_manager.c:80:18
Indirect leak of 808 byte(s) in 20 object(s) allocated from:
#0 0x5e637d7b3a4c in realloc /build/llvm-project-llvmorg-20.1.7/compiler-rt/lib/asan/asan_malloc_linux.cpp:81:3
#1 0x76e7b7330a27 in __gmp_default_reallocate (/lib/x86_64-linux-gnu/libgmp.so.10+0xda27) (BuildId: dff5c2156ec812613c5e44310
05c576b212dd323)
SUMMARY: AddressSanitizer: 73968 byte(s) leaked in 48 allocation(s).
padic_poly_pow 0.04 (PASS) 05c576b212dd323)
padic_poly_shift_left_right...
padic_poly_shift_left_right 0.01 (PASS)
padic_poly_sub...
make: *** [Makefile:776: build/fq_default_poly_factor/test/main_TEST_RUN] Error 1
make: *** Waiting for unfinished jobs....
padic_poly_sub 0.03 (PASS)