benchbase icon indicating copy to clipboard operation
benchbase copied to clipboard

goodput higher than throughput

Open miniHive opened this issue 10 months ago • 2 comments

We have repeatedly observed goodput higher than throughput (for TPC-H and YCSB on PostgreSQL);

{ "Start timestamp (milliseconds)":1739569996066, "Current Timestamp (milliseconds)":1739572194220, "Elapsed Time (nanoseconds)":2198000068342, "DBMS Type":"POSTGRES", "DBMS Version":"PostgreSQL 17.3 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit", "Benchmark Type":"tpch", "Final State":"EXIT", "Measured Requests":22, "isolation":"TRANSACTION_SERIALIZABLE", "scalefactor":"1", "terminals":"1", "Latency Distribution":{ "95th Percentile Latency (microseconds)":8171213, "Maximum Latency (microseconds)":1046028244, "Median Latency (microseconds)":1145302, "Minimum Latency (microseconds)":47069, "25th Percentile Latency (microseconds)":381294, "90th Percentile Latency (microseconds)":5461144, "99th Percentile Latency (microseconds)":1046028244, "75th Percentile Latency (microseconds)":2895255, "Average Latency (microseconds)":49201791 }, "Throughput (requests/second)":0.010009098869862678, "Goodput (requests/second)":0.020018197739725355 }

This seems counter-intuitive.

miniHive avatar Feb 15 '25 13:02 miniHive

Thanks. I've seen that before too and haven't looked into it yet. Will try and add it to the queue. Also happy to take PRs in case you get to it before me.

bpkroth avatar Feb 18 '25 15:02 bpkroth

I have observed this before too. I read the implementation and noticed that the calculation of goodput includes the requests processed during the warmup phase.

@miniHive You may want to check if that is the case? A simple check involves just multiplying throughput by the duration of (WARMUP + MEASURE) phases, and comparing that with the goodput and see if they roughly correspond.

I was confused but thought it was somehow intended.

dwslim avatar Feb 21 '25 09:02 dwslim