Nim
Nim copied to clipboard
Published
20 hours ago •
nim-lang
nim-lang
Asyncdispatch leaks under --mm:arc
Description
I opened a nim forum thread about this yesterday. It appears that using async creates a circular data-structure with its futures and that leads to leaks under arc. These become nicely visible when using address sanitizers.
I quote what I learned about this from the thread:
The problem here appears to be a cycle. I don't fully understand that myself, but leorize tried to explain it to me: The asyncCheck callback appears to construct a closure-callback, which holds on to a ref to the future being constructed by stuff(). The callback is then placed into the future itself (?), causing a cycle that arc can no longer deal with.
the worst problem in asyncdispatch is that every async call creates a cyclic reference between the future and the closure environment of the iterator whose body is generated by the async macro
Below a minimal example that demonstrates the problem:
# thefile.nim
import std/asyncdispatch
var count: int
proc stuff() {.async.} =
echo count, 1
await sleepAsync(500)
echo count, 2
count.inc
for _ in 0..<10:
asyncCheck stuff()
while hasPendingOperations(): poll()
doAssert count == 10
Compile the above with this command:
nim r --panics:on --define:danger --mm:arc --cc:clang --debugger:native --threads:on -d:butlerThreading -d:chronicles_enabled=off -d:useMalloc --debuginfo:on --linedir:on --passC:'-fno-omit-frame-pointer' --passC:'-mno-omit-leaf-frame-pointer' --passC:"-fsanitize=address" --passL:"-fsanitize=address" thefile.nim
Nim Version
Nim Compiler Version 2.0.2 [Linux: amd64] Compiled at 2023-12-15 Copyright (c) 2006-2023 by Andreas Rumpf
git hash: c4c44d10df8a14204a75c34e499def200589cb7c active boot switches: -d:release
Replicated also on:
Nim Compiler Version 2.1.1 [Linux: amd64] Compiled at 2024-01-14 Copyright (c) 2006-2024 by Andreas Rumpf
git hash: ab4278d2179639f19967431a7aa1be858046f7a7 active boot switches: -d:release
Current Output
=================================================================
==38206==ERROR: LeakSanitizer: detected memory leaks
Indirect leak of 720 byte(s) in 10 object(s) allocated from:
#0 0x55bb1ac5dcc1 in __interceptor_calloc (/home/philipp/.cache/nim/playground_r/playground_BAE78A16A6FDEA0E7101B2FCA8902C5FEC652390+0x10ecc1) (BuildId: 101922d28e23ebf8139937838107a944a4c1ccfd)
#1 0x55bb1aca7763 in alloc0Impl__system_u1763 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:11:11
#2 0x55bb1aca7763 in allocShared0Impl__system_u1776 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:37:11
#3 0x55bb1aca7763 in alignedAlloc0__system_u1941 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/memalloc.nim:351:12
#4 0x55bb1aca7763 in nimNewObj /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/arc.nim:81:24
#5 0x55bb1acb2da2 in newFuture__pureZasyncdispatch_u5397 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/asyncfutures.nim:113:70
#6 0x55bb1acc0472 in stuff__playground_u10 /home/philipp/dev/playground/src/playground.nim:5:23
#7 0x55bb1acc07b1 in NimMainModule /home/philipp/dev/playground/src/playground.nim:11:172
#8 0x55bb1acc08f1 in NimMainInner /home/philipp/dev/playground/src/playground.nim:29:2
#9 0x55bb1acc08f1 in NimMain /home/philipp/dev/playground/src/playground.nim:40:2
#10 0x55bb1acc08f1 in main /home/philipp/dev/playground/src/playground.nim:48:2
#11 0x7ff62eb9accf (/usr/lib/libc.so.6+0x27ccf) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658)
Indirect leak of 640 byte(s) in 10 object(s) allocated from:
#0 0x55bb1ac5dcc1 in __interceptor_calloc (/home/philipp/.cache/nim/playground_r/playground_BAE78A16A6FDEA0E7101B2FCA8902C5FEC652390+0x10ecc1) (BuildId: 101922d28e23ebf8139937838107a944a4c1ccfd)
#1 0x55bb1aca7763 in alloc0Impl__system_u1763 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:11:11
#2 0x55bb1aca7763 in allocShared0Impl__system_u1776 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:37:11
#3 0x55bb1aca7763 in alignedAlloc0__system_u1941 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/memalloc.nim:351:12
#4 0x55bb1aca7763 in nimNewObj /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/arc.nim:81:24
#5 0x55bb1acc0492 in stuff__playground_u10 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/asyncmacro.nim:293:102
#6 0x55bb1acc07b1 in NimMainModule /home/philipp/dev/playground/src/playground.nim:11:172
#7 0x55bb1acc08f1 in NimMainInner /home/philipp/dev/playground/src/playground.nim:29:2
#8 0x55bb1acc08f1 in NimMain /home/philipp/dev/playground/src/playground.nim:40:2
#9 0x55bb1acc08f1 in main /home/philipp/dev/playground/src/playground.nim:48:2
#10 0x7ff62eb9accf (/usr/lib/libc.so.6+0x27ccf) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658)
Indirect leak of 480 byte(s) in 10 object(s) allocated from:
#0 0x55bb1ac5dcc1 in __interceptor_calloc (/home/philipp/.cache/nim/playground_r/playground_BAE78A16A6FDEA0E7101B2FCA8902C5FEC652390+0x10ecc1) (BuildId: 101922d28e23ebf8139937838107a944a4c1ccfd)
#1 0x55bb1aca7763 in alloc0Impl__system_u1763 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:11:11
#2 0x55bb1aca7763 in allocShared0Impl__system_u1776 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:37:11
#3 0x55bb1aca7763 in alignedAlloc0__system_u1941 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/memalloc.nim:351:12
#4 0x55bb1aca7763 in nimNewObj /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/arc.nim:81:24
#5 0x55bb1acc0442 in stuff__playground_u10 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/asyncmacro.nim:232:90
#6 0x55bb1acc07b1 in NimMainModule /home/philipp/dev/playground/src/playground.nim:11:172
#7 0x55bb1acc08f1 in NimMainInner /home/philipp/dev/playground/src/playground.nim:29:2
#8 0x55bb1acc08f1 in NimMain /home/philipp/dev/playground/src/playground.nim:40:2
#9 0x55bb1acc08f1 in main /home/philipp/dev/playground/src/playground.nim:48:2
#10 0x7ff62eb9accf (/usr/lib/libc.so.6+0x27ccf) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658)
Indirect leak of 144 byte(s) in 2 object(s) allocated from:
#0 0x55bb1ac5dcc1 in __interceptor_calloc (/home/philipp/.cache/nim/playground_r/playground_BAE78A16A6FDEA0E7101B2FCA8902C5FEC652390+0x10ecc1) (BuildId: 101922d28e23ebf8139937838107a944a4c1ccfd)
#1 0x55bb1aca7763 in alloc0Impl__system_u1763 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:11:11
#2 0x55bb1aca7763 in allocShared0Impl__system_u1776 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/mm/malloc.nim:37:11
#3 0x55bb1aca7763 in alignedAlloc0__system_u1941 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/memalloc.nim:351:12
#4 0x55bb1aca7763 in nimNewObj /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/system/arc.nim:81:24
#5 0x55bb1acb2da2 in newFuture__pureZasyncdispatch_u5397 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/asyncfutures.nim:113:70
#6 0x55bb1acba14d in sleepAsync__playground_u19 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/asyncdispatch.nim:1922:14
#7 0x55bb1acbf73f in stuffX20X28AsyncX29___playground_u4 /home/philipp/dev/playground/src/playground.nim:7:29
#8 0x55bb1acbfc95 in stuffNimAsyncContinue__playground_u6 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/asyncmacro.nim:33:53
#9 0x55bb1acc058f in stuff__playground_u10 /home/philipp/.choosenim/toolchains/nim-2.0.2/lib/pure/asyncmacro.nim:293:155
#10 0x55bb1acc07b1 in NimMainModule /home/philipp/dev/playground/src/playground.nim:11:172
#11 0x55bb1acc08f1 in NimMainInner /home/philipp/dev/playground/src/playground.nim:29:2
#12 0x55bb1acc08f1 in NimMain /home/philipp/dev/playground/src/playground.nim:40:2
#13 0x55bb1acc08f1 in main /home/philipp/dev/playground/src/playground.nim:48:2
#14 0x7ff62eb9accf (/usr/lib/libc.so.6+0x27ccf) (BuildId: 8bfe03f6bf9b6a6e2591babd0bbc266837d8f658)
SUMMARY: AddressSanitizer: 1984 byte(s) leaked in 32 allocation(s).
Expected Output
No stacktraces
Possible Solution
No response
Additional Information
No response
