dmd icon indicating copy to clipboard operation
dmd copied to clipboard

Windows GC only wakes up one thread when doing parallel scanning

Open schveiguy opened this issue 7 months ago • 5 comments

In the conservative GC, the parallel threads are woken up to scan using core.sync.event. This event is set as an auto-reset event.

The way this is implemented on Posix, all threads waiting on the condition variable are awoken, and they start scanning. But the value is reset once the first thread is woken up. This does not change what happens in the GC, it sees data is ready, and scans it.

On Windows, however, an auto reset event wakes only one thread waiting on it. This means only one thread scans in parallel with the main thread, and the other threads never wake up.

The better way to do this, is to use a manually reset event, and manually reset it when the scanning is done.

Technically, this is a problem with core.sync.event, as the behavior is not the same on all platforms. But it might be difficult to change it now, either way.

schveiguy avatar Jun 04 '25 03:06 schveiguy

Honestly, I'm implementing a similar mechanism in the new GC, and I don't even understand why there is a phase of starting and stopping for the parallel threads. All the background threads should just wait on the queue for new work, and do the work as it arrives, not worrying about a starting or stopping phase. When I've printed out what is happening, strange things happen. Sometimes a scanning thread will not complete until after the main thread has started a new collection cycle, and then you get starts and stops that don't correspond to GC cycles at all.

It's very possible with the way core.sync.event is written, you can have even on posix systems threads not waking up when it's time to scan.

core.sync.event is just badly written on posix. Maybe that should be this bug report?

In any case, really both things should be changed.

schveiguy avatar Jun 04 '25 04:06 schveiguy

Your reasoning seems sound, though I'm pretty sure I saw all threads running when implementing parallel scanning.

I'll have to dig deeper again, but AFAIU there is no explicit start and stop per scan, just during GC initialization and termination. While it's running, evStart should be triggered whenever a reasonable chunk of addresses is pushed to the scanStack.

rainers avatar Jun 04 '25 16:06 rainers

While it's running, evStart should be triggered whenever a reasonable chunk of addresses is pushed to the scanStack.

That's not correct, the background threads are spinning until the scanStack is empty and no other thread is still busy scanning. Then they are waiting for a new GC cycle waiting for the evStart event.

I have tested a few versions back until 2.089, and none of them seems to use more than one background thread (though GC time tends to be small for the benchmarks anyway).

rainers avatar Jun 04 '25 19:06 rainers

see PR https://github.com/dlang/dmd/pull/21428

rainers avatar Jun 04 '25 19:06 rainers

FWIW I confirmed this by putting a sleep at the end of the gc cycle and a printout of when background threads come to life.

schveiguy avatar Jun 04 '25 21:06 schveiguy