etl
etl copied to clipboard
NDT5 upload parser under reports performance
For example reporting 4.75 Mb/s for a .pcap at 5 Mb/s (5% systematic error).
See 80.0.65.231
Systemic error in the 5% range. Not super-worrying, but should get fixed.
Possibly timers and counters are counted too early.
Deep dive into ndt-m92kv_1573028939_0000000000056D7A.pcap.gz reveals a loss recovery with 0.6 seconds of partial data in the receiver socket buffer, right at the 10s mark. This deflates the average delivery rate by about 5%. The tcpinfo measurements at 9.3s and 10.4s are within 0.5% of the rate observed by .pcap (5.000 Mb/s)
For some classes of ISPs and clients, these loss event happen at repeatable stable times. This seems to be the case with this particular client, which happens to get very low variance measurements that are consistently 5% low, because the loss is so accurately repeatable.
Details: https://console.cloud.google.com/bigquery?sq=581276032543:9d85235b691d41bab63e72731e685e75
ndt/ndt5/2019/12/28/20191228T080102.072817Z-ndt5-mlab1-lhr04-ndt.tgz ndt/pcap/2019/12/28/20191228T074635.485037Z-pcap-mlab1-lhr04-ndt.tgz ndt/tcpinfo/2019/12/28/20191228T080020.563296Z-tcpinfo-mlab1-lhr04-ndt.tgz
Dropping to a p2