tracy
tracy copied to clipboard
wolfpld
AddressSanitizer: global-buffer-overflow
I'm getting crashes from something that Address Sanitizer claims to come from within the Tracy worker thread:
==101601==ERROR: AddressSanitizer: global-buffer-overflow on address 0x0000003c8f0d at pc 0x00000053a2f7 bp 0x7f29fe9b3290 sp 0x7f29fe9b2a58
READ of size 64 at 0x0000003c8f0d thread T2
#0 0x53a2f6 in __asan_memcpy (/home/martijn/crypto/MoonBot++/build_dbg/moonbot+0x53a2f6)
#1 0xb5d515 in tracy::Profiler::AppendDataUnsafe(void const*, unsigned long) /home/martijn/crypto/MoonBot++/lib/tracy/public/common/../client/TracyProfiler.hpp:804:9
#2 0xb5d515 in tracy::Profiler::SendLongString(unsigned long, char const*, unsigned long, tracy::QueueType) /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:3032:5
#3 0xb6564d in tracy::Profiler::HandleSymbolCodeQuery(unsigned long, unsigned int) /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:3823:9
#4 0xb4d9e1 in tracy::Profiler::HandleServerQuery() /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:3397:9
#5 0xb47397 in tracy::Profiler::Worker() /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:1998:18
#6 0xbf4644 in tracy::Profiler::LaunchWorker(void*) /home/martijn/crypto/MoonBot++/lib/tracy/public/common/../client/TracyProfiler.hpp:762:63
#7 0xbf79cb in tracy::Thread::Launch(void*) /home/martijn/crypto/MoonBot++/lib/tracy/public/common/../client/TracyThread.hpp:80:40
#8 0x7f2a07c94b42 in start_thread nptl/./nptl/pthread_create.c:442:8
#9 0x7f2a07d269ff misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
0x0000003c8f0d is located 51 bytes to the left of global variable '<string literal>' defined in '/home/martijn/crypto/MoonBot++/lib/tracy/public/common/../client/TracyCallstack.hpp:127:5' (0x3c8f40) of size 11
'<string literal>' is ascii string 'depth >= 1'
0x0000003c8f0d is located 0 bytes to the right of global variable '<string literal>' defined in '/home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:677:26' (0x3c8f00) of size 13
'<string literal>' is ascii string 'RAM: %lu MB
'
SUMMARY: AddressSanitizer: global-buffer-overflow (/home/martijn/crypto/MoonBot++/build_dbg/moonbot+0x53a2f6) in __asan_memcpy
Shadow bytes around the buggy address:
0x000080071190: 00 00 06 f9 f9 f9 f9 f9 00 00 f9 f9 f9 f9 f9 f9
0x0000800711a0: 00 00 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
0x0000800711b0: 00 06 f9 f9 f9 f9 f9 f9 00 00 00 02 f9 f9 f9 f9
0x0000800711c0: 00 05 f9 f9 f9 f9 f9 f9 00 03 f9 f9 f9 f9 f9 f9
0x0000800711d0: 00 01 f9 f9 f9 f9 f9 f9 00 07 f9 f9 f9 f9 f9 f9
=>0x0000800711e0: 00[05]f9 f9 f9 f9 f9 f9 00 03 f9 f9 f9 f9 f9 f9
0x0000800711f0: 00 00 00 00 00 00 00 00 00 00 04 f9 f9 f9 f9 f9
0x000080071200: 00 00 00 04 f9 f9 f9 f9 00 00 00 00 00 00 00 00
0x000080071210: 00 00 00 00 00 00 00 00 00 00 05 f9 f9 f9 f9 f9
0x000080071220: 00 00 00 00 00 00 00 00 00 00 00 00 02 f9 f9 f9
0x000080071230: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
Thread T2 created by T0 here:
#0 0x52564c in pthread_create (/home/martijn/crypto/MoonBot++/build_dbg/moonbot+0x52564c)
#1 0xbf461b in tracy::Thread::Thread(void (*)(void*), void*) /home/martijn/crypto/MoonBot++/lib/tracy/public/common/../client/TracyThread.hpp:69:9
#2 0xb38a7a in tracy::Profiler::SpawnWorkerThreads() /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:1405:19
#3 0xb3616b in tracy::Profiler::Profiler() /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:1389:5
#4 0xbf3802 in __cxx_global_var_init.7 /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:1299:33
#5 0xbf3927 in _GLOBAL__I_000105 /home/martijn/crypto/MoonBot++/lib/tracy/public/TracyClient.cpp
#6 0x7f2a07c29eba in call_init csu/../csu/libc-start.c:145:3
#7 0x7f2a07c29eba in __libc_start_main csu/../csu/libc-start.c:379:5
==101601==ABORTING
I'm compiling this with clang++-12.
In gdb, upon ASan reporting I get:
#3 tracy::Profiler::SendLongString (this=0x1b8ca00 <tracy::s_profiler>, str=28890176, ptr=0x1b8d440 <tracy::cb_data> "p\270^\025\376\177", len=2560, type=tracy::QueueType::SymbolCode)
#4 0x0000000000b6565e in tracy::Profiler::HandleSymbolCodeQuery (this=0x1b8ca00 <tracy::s_profiler>, symbol=28890176, size=2560) at /home/martijn/crypto/MoonBot++/lib/tracy/public/client/TracyProfiler.cpp:3823
And:
(gdb) p/d TargetFrameSzie
No symbol "TargetFrameSzie" in current context.
(gdb) p/d TargetFrameSize
$9 = 262144
(gdb) p m_bufferOffset
$10 = 742341
(gdb) p m_bufferStart
$11 = 524278
(gdb) p m_bufferOffset - m_bufferStart
$12 = 218063
Looking at the source, I can't immediately figure out what's wrong, but it seems to be a problem with committing the data in time. Not committing the data doesn't reset the bufferOfset.
I think that this logic for NeedsDataSize checks if the buffer would run out of space AFTER appending the data indented. https://github.com/wolfpld/tracy/blob/f17b44c212387ae172c9c5d2b97f6989b54985fe/public/client/TracyProfiler.hpp#L791-L800 However, it doesn't check if the buffer would overflow. It seems that it would need an additional check along the lines of:
if( m_bufferOffset + (int)len > 3 * TargetFrameSize )
{
ret = CommitData();
}
3*TargetFrameSize is from:
https://github.com/wolfpld/tracy/blob/f17b44c212387ae172c9c5d2b97f6989b54985fe/public/client/TracyProfiler.cpp#L1334
Also, without understanding, I was wondering if
https://github.com/wolfpld/tracy/blob/f17b44c212387ae172c9c5d2b97f6989b54985fe/public/client/TracyProfiler.cpp#L2948
Shouldn't be using a >= comparison instead.
Aha, no, it seems that it's an overflow in reading the program code. Another run gave this:
==115014==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000000382f1f at pc 0x00000053a2f7 bp 0x7fffeda5d290 sp 0x7fffeda5ca58
READ of size 736 at 0x000000382f1f thread T2
With this as frame for SendLongString:
#3 tracy::Profiler::SendLongString (this=0x1b8d240 <tracy::s_profiler>, str=3681472,
ptr=0x382cc0, len=736, type=tracy::QueueType::SymbolCode)
I see a couple of options:
- The length of the code fragment is incorrectly determined.
- ASan is read-protecting the code sections. It always seems to communicate the read error relative to a string that sits in the program code.
Aha, no, it seems that it's an overflow in reading the program code. Another run gave this:
You can see that this is the case in your first trace:
#3 0xb6564d in tracy::Profiler::HandleSymbolCodeQuery(unsigned long, unsigned int)
The length of the code fragment is incorrectly determined.
I see the issue you are reporting only on Windows when the program shuts down and the executable images are unmapped from memory. Never during normal usage. Plus, there are some fail-safes on Linux to prevent this from surfacing.
If you believe some calculations are incorrect, it should be easy to prove it is so. The code for handling data to be sent is not particularly complicated.
ASan is read-protecting the code sections. It always seems to communicate the read error relative to a string that sits in the program code.
Well, tools are like that sometimes.
Just mentioning... I had this crash with my linux build -- enabling android's EnsureReadable fixed the issue for me
