edge icon indicating copy to clipboard operation
edge copied to clipboard
verified publisher icon storj

lock contention (?) while downloading from the gateway/linksharing with high throughput

Open amwolff opened this issue 2 years ago • 4 comments

We found out that a single AMD EPYC 7702P can only push out as much as 7.5-10 Gbps. We think that it might be possible to push out at least as much as twice more, but there's some kind of contention happening deep within the runtime (maybe caused by how we use the language?).

Flamegraph: https://pprof.host/qw0g/flamegraph

pprof host_qw0g_flamegraph

amwolff avatar Sep 05 '23 16:09 amwolff

Do we know what version of the Linux kernel was running on that box?

azdagron avatar Sep 21 '23 12:09 azdagron

And the version of Go used to compile the gateway-mt binary?

azdagron avatar Sep 21 '23 12:09 azdagron

@azdagron

  • Linux 6.2 (we also saw it with 5.19 before it got upgraded on Ubuntu 22.04 HWE)
  • Go 1.20.6

halkyon avatar Sep 26 '23 19:09 halkyon

@pwilloughby I bumped the priority of this. Feel free to grab it.

amwolff avatar Oct 05 '23 10:10 amwolff