debian12 weird performace issue with bandwidthtest-2 cpu, 4gpu, pcie4, d2h speed is not the same level

[aohan237]

i have two cpus- 8360Y 4 gpus - 4 4090 card.

two gpu attach one cpu pcie, another two attach to another cpu when use bandwidthtest the first two and the second two pci d2h have great difference. of course h2d is the same about 19Gb/s vs 25Gb/s

linux version Linux David 6.1.0-10-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.38-2 (2023-07-27) x86_64 GNU/Linux

nvidia NVIDIA-SMI 525.125.06 Driver Version: 525.125.06 CUDA Version: 12.0

i use apt to install the nvidia driver.

but when i tried ubuntu 22.04 ， the difference is gone, both can be about 24Gb/s

so what is the problem?