With `dolt_transaction_commit=1`, large latency for `START TRANSACTION` and `SET TRANSACTION ISOLATION LEVEL SERIALIZABLE`.

[coffeegoddd]

On hosted instance running Dolt v0.75.9, seeing unexpectedly large query latency times for what should be almost instant queries:

2023-04-06T17:06:37Z DEBUG [conn 1074] Query finished in 1752 ms {connectTime=2023-04-06T16:29:49Z, connectionDb=dolthubapi, query=SET TRANSACTION ISOLATION LEVEL SERIALIZABLE}

2023-04-06T17:06:37Z DEBUG [conn 1080] Query finished in 1726 ms {connectTime=2023-04-06T16:29:50Z, connectionDb=dolthubapi, query=START TRANSACTION}

These times can be as low as < 50ms, or as high as the above.

[coffeegoddd]

These large latencies disappear when dolt_transaction_commit=0. Repro is here https://github.com/dolthub/repro_tx_latency

[timsehn]

Who is helping @coffeegoddd with this? Is this blocking DoltHub on Dolt?

[coffeegoddd]

@timsehn no one is currently working on this, but it's not blocking DoltHub on Dolt. We're rolling forward with dolt_transaction_commit=0 so we don't hit this/related issues.

[timsehn]

OK. Sounds like we need a project to harden up dolt_transaction_commit=1

[timsehn]

We also noticed general instability of a Dolt server being used to serve DoltHub when dolt_transaction_commit=1. The server would crash after less than 24 hours of operation.

We need a deep dive and testing with this setting on with the goal of turning it back on for DoltHub. We'll use this issue as a general tracking issue for that work.

@PavelSafronov will own this line of work.