hyrise icon indicating copy to clipboard operation
hyrise copied to clipboard

Fix MVCC concurrency issues

Open Bouncner opened this issue 8 months ago • 9 comments

Hyrise fails in TPC-C when using link-time optimization (LTO). When putting MVCC functions (get_*/set_*) into the header (basically inlining, similar to what LTO presumably does), TPC-C also fails in the current master without LTO (see branch martin/master_with_mvcc_in_hpp).

The main issue is that we sometimes end with two visible rows of the same tuple after updating. Simply changing begin and end IDs of the MVCC data into atomics seems to help.

Unfortunately, I have not been able to create a test that fails similar to TPC-C (see random attempts in playground.cpp). Which probably shows that I still haven't understood the actual issue triggering the problem.

Changes of this PR:

  • remove several TSAN ignores which appear to actually be real issues (one boring one in the join hash, others related to MVCC)
  • fixing a concurrency issue in tpcc_random_generator.hpp
  • fix an issue with concurrent growth of ValueSegments with iterators and EntireChunkPosLists (easy to trigger in a single-threaded test without LTO; no idea why we haven't seen it before)
  • note: playground will be reverted before merging

Todos:

  • [x] Get stress test running
  • [x] Check benchmark TSAN runs on large machine
  • [x] benchmark_all.sh

Thanks to @dey4ss for the help.

Fixes #2653.

Bouncner avatar Jun 05 '24 22:06 Bouncner