Why does read latency not equal to write latency?

[oleotiger]

According to my understanding, rdma read and rdma write is the same except data flow direction. I think latency of ib_read_lat should be equal to that of ib_write_lat.

Test command: numactl --cpunodebind=0 --membind=0 ./ib_read/write_lat -d mlx5_0 -R ip_address

Result:

• Read latency : 1.91 us.
• Raw write latency : 0.93 us
• Write latency after disabling IBV_SEND_INLINE: 1.32us
• Write latency after disabling IBV_SEND_INLINE and DDIO: 1.32us

Are there any other reasons that may cause the difference?

[bsobczak]

Sorry, not sure what do you mean by write and read being the same in terms of latency?

It usually all boils down to the way the latency is really calculated, but in order to read data through RDMA the data sink server needs to send Read Request message and waits for Read Response message from data source server. Maybe there are wiser men here, who can prove me wrong, but the claim for the latency doesn't look true to me.

Also, not sure what difference do you mean? Can you specify your question?

[HassanKhadour]

Thanks @bsobczak for your answer.

I'm closing this Issue since its not a bug, Please re open if you think it is.