Confusing debug prints

[tanisman]

trafficstars

When allocating & deleting small memory blocks sequentially, some confusing debug prints appear randomly. The debug message says "pointer might not point to a valid heap region" and "this may be still valid very large allocation (over 64MiB)" but my allocations always smaller than 300 bytes.

Should I ignore this message or it is a bug or am I using the library incorrectly?

alloc 272 [0000040000030140]
alloc 16 [00000400000102E0]
alloc 16 [0000040000010300]
alloc 16 [0000040000010320]
alloc 40 [00000400000400C0]
alloc 16 [0000040000010340]
alloc 16 [0000040000010360]
alloc 16 [0000040000010380]
delete 16 [0000040000010380]
alloc 8 [0000040000001550]
alloc 4 [0000040000001560]
alloc 8 [0000040000001570]
alloc 4 [0000040000001580]
delete 64 [0000040000020050]
delete 16 [0000040000010020]
delete 64 [0000040000020000]
delete 16 [0000040000010000]
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0000040000080000
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 0000040000080000 was valid after all)
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0000040000080000
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 0000040000080000 was valid after all)
alloc 248 [0000040000080000]
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 00000400000400F0
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 00000400000400F0 was valid after all)
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 00000400000400F0
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 00000400000400F0 was valid after all)
alloc 32 [00000400000400F0]
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 00000400000103A0
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 00000400000103A0 was valid after all)
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 00000400000103A0
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 00000400000103A0 was valid after all)
alloc 16 [00000400000103A0]
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0000040000040120
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 0000040000040120 was valid after all)
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0000040000040120
> (this may still be a valid very large allocation (over 64MiB))
> mimalloc: warning: (yes, the previous pointer 0000040000040120 was valid after all)
alloc 32 [0000040000040120]
> mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 00000400000103C0
> (this may still be a valid very large allocation (over 64MiB))
alloc 16 [00000400000103C0]
delete 16 [00000400000103C0]
alloc 16 [00000400000103E0]
alloc 64 [0000040000020190]
alloc 16 [0000040000010400]
delete 16 [0000040000010400]
delete 32 [0000040000040120]
delete 16 [00000400000103A0]
delete 32 [00000400000400F0]
delete 248 [0000040000080000]
alloc 248 [0000040000080100]

[daanx]

Hmm, it is all fine (as the message confirms later) but not sure why it happens in the first place; I will look into it.

[daanx]

What mimalloc version are you running? v2.0.0 or v1.7.0 ? If neither, can you check if you get the error with either version?

[tanisman]

I use v1.7.0, will try v2.0.0 and post the results

[daanx]

Thanks -- if you have a smallish repro program that would be great :-) (apologies for letting you try multiple versions but each uses different ways to determine if a pointer belongs in a heap region)

[doublex]

Same debug log (v2.0.3) Maybe due fork()?

mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_free: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_free: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dc0020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dc0020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dc0020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dc0020000 was valid after all)
mimalloc: warning: mi_free: pointer might not point to a valid heap region: 0x7f8dc0020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dc0020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dc0020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dc0020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_free: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8dbc020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x7f8dbc020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x7f8db8020000
(this may still be a valid very large allocation (over 64MiB))

[daanx]

Hi, I pushed a potential fix to the dev-slice branch (= v2.0 versions). If you can, give it a try and see if it fixes the issue.

[koobs]

I am observing these warnings when building software (FreeBSD Ports) with mold linker (uses mimalloc). Suggestions for further isolation / debugging to assist your resolution would be great. cc @rui314

Edit: Note: The symptoms are reproducible in this environment just invoking the mold command.

Reproduction Environment

• FreeBSD 14.0-CURRENT main-n254374-4fd141c7d7a GENERIC-NODEBUG amd64
• mimalloc 2.0.5 and 2.0.6
• mold linker 1.1 and 1.2

mimalloc build details

Otherwise standard/default cmake build, modulo:

MI_INSTALL_TOPLEVEL=ON
MI_BUILD_OBJECT=OFF

CFLAGS/LDFLAGS available in Debug Log below

Reproduction Steps

• Build/install mold linker with any mimalloc version ~~- Build mimalloc with mold linker~~
  • -fuse-ld=/path/to/mold, OR
  • --ld-path=/path/to/mold
• Run mold command

Debug Log

mimalloc build with mold with warnings

[ 94% 36/37] : && /usr/bin/cc -fcolor-diagnostics -march=sandybridge -mtune=sandybridge -O2 -pipe -fcf-protection -fstack-clash-protection  -fstack-protector-strong -fpie -fpic -fPIC -fno-strict-aliasing -fcolor-diagnostics -march=sandyb
ridge -mtune=sandybridge -O2 -pipe -fcf-protection -fstack-clash-protection  -fstack-protector-strong -fpie -fpic -fPIC -fno-strict-aliasing --ld-path=/usr/local/bin/mold -fstack-protector-strong -Wl,--enable-new-dtags -z relro -z now -p
ie -z notext CMakeFiles/mimalloc-test-api.dir/test/test-api.c.o -o mimalloc-test-api  -Wl,-rpath,/tmp/tmpfs0/usr/home/koobs/repos/freebsd/ports/devel/mimalloc/work/.build  libmimalloc.so.2.0  -lpthread  -lrt && :
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x247450030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x247450020700
(this may still be a valid very large allocation (over 64MiB))
[ 97% 36/37] /usr/bin/cc  -I/tmp/tmpfs0/usr/home/koobs/repos/freebsd/ports/devel/mimalloc/work/mimalloc-2.0.6/include -fcolor-diagnostics -march=sandybridge -mtune=sandybridge -O2 -pipe -fcf-protection -fstack-clash-protection  -fstack-p
rotector-strong -fpie -fpic -fPIC -fno-strict-aliasing -fcolor-diagnostics -march=sandybridge -mtune=sandybridge -O2 -pipe -fcf-protection -fstack-clash-protection  -fstack-protector-strong -fpie -fpic -fPIC -fno-strict-aliasing -Wall -W
extra -Wno-unknown-pragmas -fvisibility=hidden -Wstrict-prototypes -Wpedantic -Wno-static-in-inline -ftls-model=initial-exec -fno-builtin-malloc -std=gnu11 -MD -MT CMakeFiles/mimalloc-test-api-fill.dir/test/test-api-fill.c.o -MF CMakeFil
es/mimalloc-test-api-fill.dir/test/test-api-fill.c.o.d -o CMakeFiles/mimalloc-test-api-fill.dir/test/test-api-fill.c.o -c /tmp/tmpfs0/usr/home/koobs/repos/freebsd/ports/devel/mimalloc/work/mimalloc-2.0.6/test/test-api-fill.c
[100% 37/37] : && /usr/bin/cc -fcolor-diagnostics -march=sandybridge -mtune=sandybridge -O2 -pipe -fcf-protection -fstack-clash-protection  -fstack-protector-strong -fpie -fpic -fPIC -fno-strict-aliasing -fcolor-diagnostics -march=sandyb
ridge -mtune=sandybridge -O2 -pipe -fcf-protection -fstack-clash-protection  -fstack-protector-strong -fpie -fpic -fPIC -fno-strict-aliasing --ld-path=/usr/local/bin/mold -fstack-protector-strong -Wl,--enable-new-dtags -z relro -z now -p
ie -z notext CMakeFiles/mimalloc-test-api-fill.dir/test/test-api-fill.c.o -o mimalloc-test-api-fill  -Wl,-rpath,/tmp/tmpfs0/usr/home/koobs/repos/freebsd/ports/devel/mimalloc/work/.build  libmimalloc.so.2.0  -lpthread  -lrt && :
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec020000
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec020000 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec030080
(this may still be a valid very large allocation (over 64MiB))
mimalloc: warning: (yes, the previous pointer 0x3542ec030080 was valid after all)
mimalloc: warning: mi_usable_size: pointer might not point to a valid heap region: 0x3542ec020700
(this may still be a valid very large allocation (over 64MiB))
===>>> Building the port required 11 seconds

[devnexen]

I tried to reproduce with a slightly more advanced freebsd 14 snapshot, I could not ... did you compile mold from source or did you use the port ?

[koobs]

I tried to reproduce with a slightly more advanced freebsd 14 snapshot, I could not ... did you compile mold from source or did you use the port ?

Using the FreeBSD mold and mimalloc ports on the latest (main) revisions (mimalloc recently updated to 2.0.6).

The debug messages don't appear to be displayed 100% of the time. Out of N (~10) mold invocations, they are visible at ~50-75% rates.

[devnexen]

no luck so far .. I spent time trying to build several times mimalloc with -fuse-ld case then --ld-path still can t reproduce.

[LinuxUserGD]

I am observing these warnings when building software (FreeBSD Ports) with mold linker (uses mimalloc). Suggestions for further isolation / debugging to assist your resolution would be great. cc @rui314

Steps to reproduce (Edit: fixed by rui314/mold@da3f5dd)

Compile mold, tbb and mimalloc with clang, -stdlib=libc++ and -fuse-ld=lld

• Recompile sys-libs/musl with mold linker and -flto on Gentoo Linux (clang toolchain with libc++ stdlib).

Compiling any package afterwards with -flto and mold will fail at build process (e. g. dev-util/b2) with

> clang++  -O3 -pipe -march=native -mtune=native -D_FORTIFY_SOURCE=3 -g0 -flto -stdlib=libc++  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -O3 -pipe -march=native -mtune=native -D_FORTIFY_SOURCE=3 -g0 -flto  -Wl,-O3 -Wl,--as-needed -Wl,--strip-debug -Wl,--undefined-version -Wl,--icf=safe -Wl,--thread-count=4 -Wl,--compress-debug-sections=zstd -fuse-ld=mold -rtlib=compiler-rt -unwindlib=libunwind check_cxx11.cpp
mimalloc: error: mi_free: pointer does not point to a valid heap space: 0x7f4dcb8f1980
clang++: error: unable to execute command: Segmentation fault (core dumped)
clang++: error: linker command failed due to signal (use -v to see invocation)
> clang++  -O3 -pipe -march=native -mtune=native -D_FORTIFY_SOURCE=3 -g0 -flto -stdlib=libc++  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -O3 -pipe -march=native -mtune=native -D_FORTIFY_SOURCE=3 -g0 -flto  -Wl,-O3 -Wl,--as-needed -Wl,--strip-debug -Wl,--undefined-version -Wl,--icf=safe -Wl,--thread-count=4 -Wl,--compress-debug-sections=zstd -fuse-ld=mold -rtlib=compiler-rt -unwindlib=libunwind check_cxx11.cpp
mimalloc: error: mi_free: pointer does not point to a valid heap space: 0x7f9f1f0e3980
clang++: error: unable to execute command: Segmentation fault (core dumped)
clang++: error: linker command failed due to signal (use -v to see invocation)
> clang++  -O3 -pipe -march=native -mtune=native -D_FORTIFY_SOURCE=3 -g0 -flto -stdlib=libc++  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -O3 -pipe -march=native -mtune=native -D_FORTIFY_SOURCE=3 -g0 -flto  -Wl,-O3 -Wl,--as-needed -Wl,--strip-debug -Wl,--undefined-version -Wl,--icf=safe -Wl,--thread-count=4 -Wl,--compress-debug-sections=zstd -fuse-ld=mold -rtlib=compiler-rt -unwindlib=libunwind check_cxx11.cpp
mimalloc: error: mi_free: pointer does not point to a valid heap space: 0x7fe161655980
clang++: error: unable to execute command: Segmentation fault (core dumped)
clang++: error: linker command failed due to signal (use -v to see invocation)