From Discord: Memory Leak in Effect: `zipLatest` with `async` Stream Causing Fiber Overflow

[effect-bot]

Summary

Summary:

@avaq_ reported an issue with the Effect library where an ever-increasing number of fibers are spawned, leading to a memory leak. This occurs when combining two streams using zipLatest, where one stream is created with async and the other never emits a value. @avaq_ provided a minimal reproduction script to demonstrate the issue.

@datner_ confirmed the issue, noting that the fiber count increases until the first emit of the non-emitting stream. After this point, the number of fibers stabilizes. @datner_ also suggested that the issue might not occur with .fromSchedule because it doesn't create threads, whereas emit does, leading to fiber creation.

Key Takeaways:

1. Issue Identification: Combining an async stream with a non-emitting stream using zipLatest leads to a memory leak due to an ever-increasing number of fibers.
2. Reproduction: The issue can be reproduced using a minimal script provided by @avaq_.
3. Behavior Observation: The fiber count increases until the non-emitting stream emits for the first time, after which it stabilizes.
4. Potential Cause: The use of emit in async creates fibers, unlike .fromSchedule, which might be why the issue doesn't occur with the latter.

Discord thread

https://discord.com/channels/795981131316985866/1259908822496055347

[datner]

I can verify. This happens in general as long we're before the first emit of the other stream. So it would accumulate fibers and once the other stream emits all the fibers would clean up. It seems that once over that threshold, the fibers are kept at a constant 4-5 regardless of the rate of either streams which is odd. I believe that the reason you don't see this with .fromSchedule is because it doesn't create fibers, while emit obviously does (you exit and re-enter effect, this action alone creates a fiber)

playground reproduction