ydb icon indicating copy to clipboard operation
ydb copied to clipboard

Fix the aborting transactions in Column Tables

Open kirillvasilenko opened this issue 1 month ago • 6 comments

fixes #27113

In a nutshell:

  1. Added a fairly strict lifecycle for LockFeature NeedsAborting -> Aborting -> Aborted
  2. Unified how we process aborts, now there are 3 ways:
    1. Abort transaction before kqp sends EvWrite Prepare. The entry point is ColumnShard::TransactionToAbort() method. It covers the following events:
      1. SubscribeLock -> EvLockStatus Unavailable
      2. EvWrite Rollback, from kqp
      3. TxBlobsWritingFailed
      4. In flight locks ranges bytes limit exceeded
    2. Cancel the proposed transaction. After kqp sends EvWrit Prepare but before coordinator sends PlanStep. The entry point is TColumnShard::CancelTransaction(). It covers the following events:
      1. TEvCancelTransactionProposal, from kqp
      2. The deadline for a transaction comes, but coordinator has not yet sent PlanStep. We cancel this transaction.
    3. After coordinator sends PlanStep. Distributed commit decides that the transaction must be aborted.
  3. Made the abort be executed only once
  4. Made SubscribeLock happen only once
  5. Fixed loading the lock state
  6. Deleted AddTemporaryTxLink()
  7. Stopped calling cancel when a transaction receives ReadSet with the broken flag. That method never executed, it requires PlanStep == 0, but at the end of the distributed commit PlanStep is never 0.

kirillvasilenko avatar Dec 10 '25 13:12 kirillvasilenko

:white_circle: 2025-12-10 13:27:38 UTC Pre-commit check linux-x86_64-release-asan for f7c19a72aa1f2cc2351e18a82772c5b2082d3995 has started. :white_circle: 2025-12-10 13:28:18 UTC Artifacts will be uploaded here :white_circle: 2025-12-10 13:30:12 UTC ya make is running... :yellow_circle: 2025-12-10 14:35:13 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11740 11660 0 63 7 10

:green_circle: 2025-12-10 14:35:22 UTC Build successful. :green_circle: 2025-12-10 14:35:53 UTC ydbd size 3.8 GiB changed* by -576 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 30bfd7f172aeba61f47c97a0ab531beb0d8e0718 merge: f7c19a72aa1f2cc2351e18a82772c5b2082d3995 diff diff %
ydbd size 4 132 943 200 Bytes 4 132 942 624 Bytes -576 Bytes -0.000%
ydbd stripped size 1 534 071 288 Bytes 1 534 071 224 Bytes -64 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 10 '25 13:12 github-actions[bot]

Run Extra Tests

Run additional tests for this PR. You can customize:

  • Test Size: small, medium, large (default: all)
  • Test Targets: any directory path (default: ydb/)
  • Sanitizers: ASAN, MSAN, TSAN
  • Coredumps: enable for debugging (default: off)
  • Additional args: custom ya make arguments

▶  Run tests

ydbot avatar Dec 10 '25 13:12 ydbot

:green_circle: 2025-12-11 17:57:40 UTC The validation of the Pull Request description is successful.

github-actions[bot] avatar Dec 10 '25 13:12 github-actions[bot]

:white_circle: 2025-12-10 13:29:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for f7c19a72aa1f2cc2351e18a82772c5b2082d3995 has started. :white_circle: 2025-12-10 13:29:56 UTC Artifacts will be uploaded here :white_circle: 2025-12-10 13:32:42 UTC ya make is running... :yellow_circle: 2025-12-10 15:07:37 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
32396 30949 0 9 1417 21

:white_circle: 2025-12-10 15:07:50 UTC ya make is running... (failed tests rerun, try 2) :yellow_circle: 2025-12-10 15:27:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
54 (only retried tests) 37 0 1 0 16

:white_circle: 2025-12-10 15:27:59 UTC ya make is running... (failed tests rerun, try 3) :red_circle: 2025-12-10 15:47:07 UTC Some tests failed, follow the links below.

Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
30 (only retried tests) 14 0 1 0 15

:green_circle: 2025-12-10 15:47:15 UTC Build successful. :green_circle: 2025-12-10 15:47:43 UTC ydbd size 2.3 GiB changed* by -112 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash main: 30bfd7f172aeba61f47c97a0ab531beb0d8e0718 merge: f7c19a72aa1f2cc2351e18a82772c5b2082d3995 diff diff %
ydbd size 2 468 031 736 Bytes 2 468 031 624 Bytes -112 Bytes -0.000%
ydbd stripped size 525 181 120 Bytes 525 181 120 Bytes 0 Bytes 0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 10 '25 13:12 github-actions[bot]

:white_circle: 2025-12-11 13:12:03 UTC Pre-commit check linux-x86_64-release-asan for 9b332274f25a4e50fcc09e61a58fbf592cfabfd4 has started. :white_circle: 2025-12-11 13:12:21 UTC Artifacts will be uploaded here :white_circle: 2025-12-11 13:14:33 UTC ya make is running... :yellow_circle: 2025-12-11 14:24:45 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12044 11935 0 93 7 9

:green_circle: 2025-12-11 14:24:53 UTC Build successful. :green_circle: 2025-12-11 14:25:26 UTC ydbd size 3.8 GiB changed* by +27.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4eb6709e36f24021c00a189027293d1194b29863 merge: 9b332274f25a4e50fcc09e61a58fbf592cfabfd4 diff diff %
ydbd size 4 133 680 200 Bytes 4 133 708 048 Bytes +27.2 KiB +0.001%
ydbd stripped size 1 534 344 728 Bytes 1 534 344 952 Bytes +224 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 11 '25 13:12 github-actions[bot]

:white_circle: 2025-12-11 13:12:45 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9b332274f25a4e50fcc09e61a58fbf592cfabfd4 has started. :white_circle: 2025-12-11 13:13:02 UTC Artifacts will be uploaded here :white_circle: 2025-12-11 13:15:10 UTC ya make is running... :yellow_circle: 2025-12-11 14:57:39 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40208 37113 0 164 2827 104

:white_circle: 2025-12-11 14:57:52 UTC ya make is running... (failed tests rerun, try 2) :green_circle: 2025-12-11 15:18:20 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
291 (only retried tests) 197 0 0 76 18

:green_circle: 2025-12-11 15:18:27 UTC Build successful. :green_circle: 2025-12-11 15:18:52 UTC ydbd size 2.3 GiB changed* by +25.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4eb6709e36f24021c00a189027293d1194b29863 merge: 9b332274f25a4e50fcc09e61a58fbf592cfabfd4 diff diff %
ydbd size 2 468 498 464 Bytes 2 468 524 272 Bytes +25.2 KiB +0.001%
ydbd stripped size 525 298 688 Bytes 525 299 456 Bytes +768 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 11 '25 13:12 github-actions[bot]

:white_circle: 2025-12-12 14:19:50 UTC Pre-commit check linux-x86_64-relwithdebinfo for 05ee7a2989838a13e9a4ca8d4bedf58183c6ecbc has started. :white_circle: 2025-12-12 14:19:55 UTC Artifacts will be uploaded here :white_circle: 2025-12-12 14:21:19 UTC ya make is running... :yellow_circle: 2025-12-12 16:04:28 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40236 37302 0 5 2902 27

:white_circle: 2025-12-12 16:04:45 UTC ya make is running... (failed tests rerun, try 2) :green_circle: 2025-12-12 16:17:51 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
51 (only retried tests) 37 0 0 0 14

:green_circle: 2025-12-12 16:17:58 UTC Build successful. :green_circle: 2025-12-12 16:18:21 UTC ydbd size 2.3 GiB changed* by +23.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: b318bf4e91082d97ac3d87bfc16fe090bf97091d merge: 05ee7a2989838a13e9a4ca8d4bedf58183c6ecbc diff diff %
ydbd size 2 468 914 008 Bytes 2 468 937 576 Bytes +23.0 KiB +0.001%
ydbd stripped size 525 348 768 Bytes 525 349 088 Bytes +320 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 12 '25 14:12 github-actions[bot]

:white_circle: 2025-12-12 14:21:23 UTC Pre-commit check linux-x86_64-release-asan for 05ee7a2989838a13e9a4ca8d4bedf58183c6ecbc has started. :white_circle: 2025-12-12 14:21:41 UTC Artifacts will be uploaded here :white_circle: 2025-12-12 14:23:56 UTC ya make is running... :yellow_circle: 2025-12-12 15:49:56 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12069 11695 0 328 33 13

:green_circle: 2025-12-12 15:50:06 UTC Build successful. :green_circle: 2025-12-12 15:50:40 UTC ydbd size 3.9 GiB changed* by +27.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: b318bf4e91082d97ac3d87bfc16fe090bf97091d merge: 05ee7a2989838a13e9a4ca8d4bedf58183c6ecbc diff diff %
ydbd size 4 134 385 336 Bytes 4 134 413 392 Bytes +27.4 KiB +0.001%
ydbd stripped size 1 534 535 640 Bytes 1 534 538 680 Bytes +3.0 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 12 '25 14:12 github-actions[bot]

Backport

To backport this PR, click the button next to the target branch and then click "Run workflow" in the Run Actions UI.

Branch Run
stable-25-2, stable-25-2-1, stable-25-3, stable-25-3-1 ▶  Backport
stable-25-3, stable-25-3-1 ▶  Backport
stable-25-3 ▶  Backport

▶  Backport manual

ydbot avatar Dec 15 '25 13:12 ydbot