Michael Pratt

Some thoughts from brainstorming for posterity: My best theory at the moment (though I'd really like to see perf to confirm) is that ~90 threads are calling epoll_wait at once...

@bwerthmann Thanks for the detailed profile! This seems to confirm my suspicion in https://github.com/golang/go/issues/65064#issuecomment-1887902738. The mutex being taken appears to be https://elixir.bootlin.com/linux/v5.10.209/source/fs/eventpoll.c#L696 [1]. This is held around a loop over...

@JetSetIlly Are you seeing differences without PGO? I've run your reproducer on my machine (thanks for making it easy to use!), and this is what I see: ``` │ go1.21.7.nopgo.txt...

Interesting, that is quite different from what I am seeing. The regression and variance disappears when you revert https://go.dev/cl/501976? Could you share runtime execution traces from these runs, both with...

Ah, by execution traces, I meant traces from [`runtime/trace`](https://pkg.go.dev/runtime/trace), not CPU profiles. Sorry for the confusion.

When you say 'CPU in the scheduler', I assume you mean the host Linux kernel scheduler? Which platform (or platforms) do you see this behavior on?

For reference, this workload tends to suffer from https://github.com/golang/go/issues/43997.

Note that this bug is not related to rootless. It fails the same way when run normally.

Sounds good, thanks! I'll be sure to measure the performance. > I was going to ask whether the switch to hardcode `llvm-symbolizer-14` means older version support would break but then...

Does this code need to use fastrand at all? As far I can tell, `sync.Rand*` are used in only 2 places, both of which are one-time seed initialization. Performance shouldn't...