Investigate impact of increasing timestamp_history_retention_interval_sec gflag

[ndeodhar]

trafficstars

Jira Link: DB-1856 Currently, timestamp_history_retention_interval_sec gflag is set to 2mins. This means that we can't support transactions running longer than 2 mins. In YSQL relational workloads, it's not uncommon to have long running transactions. Increasing this gflag could have a performance impact (especially for update heavy workloads) since this gflag dictates what rows get compacted.

We should run some perf tests and find a reasonable limit for this gflag. Lets run the tests for gflag values 10 , 15, 30 and 60 mins and measure the impact on TPCC run and sysbench update workloads.

[rthallamko3]

@robertsami , Is the above valid? Don't we hold on to WALs corresponding to active transactions?

[kmuthukk]

I think this issue is stale/should be closed now. A lot of work has been done to support long running transactions now.

Also, <<Don't we hold on to WALs corresponding to active transactions?>> I think WAL retention is independent of the "history_retension_interval_sec" knob. The MVCC retention applies to data flushed data (SSTable files) where we keep enough older versions to respect this retention interval. This should not block the older WAL logs from getting garbage collected.

[rthallamko3]

Thanks @kmuthukk , closing this issue.