ydb icon indicating copy to clipboard operation
ydb copied to clipboard

DataShard: support snapshot isolation reads and writes

Open snaury opened this issue 1 month ago • 12 comments

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

Support for snapshot isolation operations in datashard. Fixes #12971.

snaury avatar Dec 11 '25 09:12 snaury

:white_circle: 2025-12-11 09:30:55 UTC Pre-commit check linux-x86_64-release-asan for a57f4b18b44ee6528ec7bcfc2aaacaee6ea201d5 has started. :white_circle: 2025-12-11 09:31:13 UTC Artifacts will be uploaded here :white_circle: 2025-12-11 09:33:30 UTC ya make is running... :black_circle: 2025-12-11 10:20:46 UTC Check cancelled

github-actions[bot] avatar Dec 11 '25 09:12 github-actions[bot]

Run Extra Tests

Run additional tests for this PR. You can customize:

  • Test Size: small, medium, large (default: all)
  • Test Targets: any directory path (default: ydb/)
  • Sanitizers: ASAN, MSAN, TSAN
  • Coredumps: enable for debugging (default: off)
  • Additional args: custom ya make arguments

▶  Run tests

ydbot avatar Dec 11 '25 09:12 ydbot

:green_circle: 2025-12-11 09:32:29 UTC The validation of the Pull Request description is successful.

github-actions[bot] avatar Dec 11 '25 09:12 github-actions[bot]

:white_circle: 2025-12-11 09:33:13 UTC Pre-commit check linux-x86_64-relwithdebinfo for a57f4b18b44ee6528ec7bcfc2aaacaee6ea201d5 has started. :white_circle: 2025-12-11 09:33:30 UTC Artifacts will be uploaded here :white_circle: 2025-12-11 09:35:43 UTC ya make is running... :black_circle: 2025-12-11 10:20:55 UTC Check cancelled

github-actions[bot] avatar Dec 11 '25 09:12 github-actions[bot]

:white_circle: 2025-12-11 10:23:25 UTC Pre-commit check linux-x86_64-relwithdebinfo for e5a5af6e0a273ca1adb67c177c3f870db40c5f2a has started. :white_circle: 2025-12-11 10:23:41 UTC Artifacts will be uploaded here :white_circle: 2025-12-11 10:25:59 UTC ya make is running... :yellow_circle: 2025-12-11 12:34:55 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41372 38434 0 6 2904 28

:white_circle: 2025-12-11 12:35:09 UTC ya make is running... (failed tests rerun, try 2) :green_circle: 2025-12-11 12:48:51 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
54 (only retried tests) 40 0 0 0 14

:green_circle: 2025-12-11 12:48:57 UTC Build successful. :green_circle: 2025-12-11 12:49:22 UTC ydbd size 2.3 GiB changed* by +9.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 00bc17a837c60b469f190935a0e10f038f1746f7 merge: e5a5af6e0a273ca1adb67c177c3f870db40c5f2a diff diff %
ydbd size 2 468 434 248 Bytes 2 468 443 696 Bytes +9.2 KiB +0.000%
ydbd stripped size 525 274 816 Bytes 525 278 048 Bytes +3.2 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 11 '25 10:12 github-actions[bot]

:white_circle: 2025-12-11 10:23:39 UTC Pre-commit check linux-x86_64-release-asan for e5a5af6e0a273ca1adb67c177c3f870db40c5f2a has started. :white_circle: 2025-12-11 10:23:56 UTC Artifacts will be uploaded here :white_circle: 2025-12-11 10:26:07 UTC ya make is running... :yellow_circle: 2025-12-11 12:05:49 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12957 12865 0 71 12 9

:green_circle: 2025-12-11 12:06:01 UTC Build successful. :green_circle: 2025-12-11 12:06:35 UTC ydbd size 3.8 GiB changed* by +26.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 00bc17a837c60b469f190935a0e10f038f1746f7 merge: e5a5af6e0a273ca1adb67c177c3f870db40c5f2a diff diff %
ydbd size 4 133 529 896 Bytes 4 133 556 944 Bytes +26.4 KiB +0.001%
ydbd stripped size 1 534 259 224 Bytes 1 534 275 320 Bytes +15.7 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 11 '25 10:12 github-actions[bot]

May be add more tests?

Я специально не использую в тестах KQP, и не тестирую транзакционное поведение. Вместо этого я выполняю конкретные операции, и тестирую поведение конкретных TEvWrite операций.

  1. Read-Read Conflicts

Я уже тестирую, что чтение не берёт локи. Консистентность чтений никак не меняется.

  1. Write Skew Anomaly

Я уже тестирую, что запись в ключ детектит изменения от других транзакций и абортится, а если изменение в другой ключ - не абортится. Мне здесь не нужно тестировать write skew, потому что я делаю запчасть для реализации транзакций, а не сами транзакции, и проверяю контракт этих операций.

  1. Range Operations

Я не вижу смысла в этих тестах выполнять range операции просто так, потому что я в них ничего не менял. Там есть разве что момент с TEvReadContinue после того как лок сломался, и детектом неконсистентности, это наверное можно отдельно протестировать, потому что оно должно успешно читать то, на что serializable бы сломал себе лок.

  1. Rollback

Незакомиченные изменения под локом пишутся также как писались, тесты на откат изменений уже есть, я не вижу смысла их повторять.

snaury avatar Dec 12 '25 09:12 snaury

:white_circle: 2025-12-12 10:19:10 UTC Pre-commit check linux-x86_64-relwithdebinfo for 68b8da626a9f5baf15548aba18d9fb55f7f9b8c0 has started. :white_circle: 2025-12-12 10:19:30 UTC Artifacts will be uploaded here :white_circle: 2025-12-12 10:21:39 UTC ya make is running... :yellow_circle: 2025-12-12 12:21:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41400 38435 0 18 2918 29

:white_circle: 2025-12-12 12:22:05 UTC ya make is running... (failed tests rerun, try 2) :green_circle: 2025-12-12 12:28:08 UTC ydbd size 2.3 GiB changed* by +8.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 7fe9277b79b8d3dcbcd81a3e3c3d870e6c3b7812 merge: 68b8da626a9f5baf15548aba18d9fb55f7f9b8c0 diff diff %
ydbd size 2 468 806 360 Bytes 2 468 815 448 Bytes +8.9 KiB +0.000%
ydbd stripped size 525 344 608 Bytes 525 347 904 Bytes +3.2 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation :black_circle: 2025-12-12 12:28:10 UTC Check cancelled

github-actions[bot] avatar Dec 12 '25 10:12 github-actions[bot]

:white_circle: 2025-12-12 10:19:12 UTC Pre-commit check linux-x86_64-release-asan for 68b8da626a9f5baf15548aba18d9fb55f7f9b8c0 has started. :white_circle: 2025-12-12 10:19:28 UTC Artifacts will be uploaded here :white_circle: 2025-12-12 10:21:34 UTC ya make is running... :yellow_circle: 2025-12-12 11:57:17 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12983 12873 0 91 7 12

:green_circle: 2025-12-12 11:57:26 UTC Build successful. :green_circle: 2025-12-12 11:57:57 UTC ydbd size 3.9 GiB changed* by +21.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 7fe9277b79b8d3dcbcd81a3e3c3d870e6c3b7812 merge: 68b8da626a9f5baf15548aba18d9fb55f7f9b8c0 diff diff %
ydbd size 4 134 217 592 Bytes 4 134 239 560 Bytes +21.5 KiB +0.001%
ydbd stripped size 1 534 515 928 Bytes 1 534 527 544 Bytes +11.3 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 12 '25 10:12 github-actions[bot]

Добавил ещё тесты с range read'ами и исправил ошибку чтения в одном из кейсов.

snaury avatar Dec 12 '25 12:12 snaury

:white_circle: 2025-12-12 12:34:35 UTC Pre-commit check linux-x86_64-release-asan for 88c46ed0105d7bd04b0a4a075755bfb0f84eda8d has started. :white_circle: 2025-12-12 12:36:17 UTC Artifacts will be uploaded here :white_circle: 2025-12-12 12:38:23 UTC ya make is running... :yellow_circle: 2025-12-12 14:12:37 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12432 12274 0 138 7 13

:green_circle: 2025-12-12 14:12:48 UTC Build successful. :green_circle: 2025-12-12 14:13:21 UTC ydbd size 3.9 GiB changed* by +17.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 3c7edea4dc2f22301e0f3f054b92eddc5b67ce91 merge: 88c46ed0105d7bd04b0a4a075755bfb0f84eda8d diff diff %
ydbd size 4 134 225 024 Bytes 4 134 242 880 Bytes +17.4 KiB +0.000%
ydbd stripped size 1 534 497 496 Bytes 1 534 504 952 Bytes +7.3 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 12 '25 12:12 github-actions[bot]

:white_circle: 2025-12-12 12:36:14 UTC Pre-commit check linux-x86_64-relwithdebinfo for 88c46ed0105d7bd04b0a4a075755bfb0f84eda8d has started. :white_circle: 2025-12-12 12:36:21 UTC Artifacts will be uploaded here :white_circle: 2025-12-12 12:38:37 UTC ya make is running... :yellow_circle: 2025-12-12 14:33:30 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40598 37646 0 4 2918 30

:white_circle: 2025-12-12 14:33:48 UTC ya make is running... (failed tests rerun, try 2) :green_circle: 2025-12-12 14:47:06 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
84 (only retried tests) 66 0 0 0 18

:green_circle: 2025-12-12 14:47:21 UTC Build successful. :green_circle: 2025-12-12 14:47:47 UTC ydbd size 2.3 GiB changed* by +8.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 3c7edea4dc2f22301e0f3f054b92eddc5b67ce91 merge: 88c46ed0105d7bd04b0a4a075755bfb0f84eda8d diff diff %
ydbd size 2 468 811 232 Bytes 2 468 820 376 Bytes +8.9 KiB +0.000%
ydbd stripped size 525 338 784 Bytes 525 342 080 Bytes +3.2 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

github-actions[bot] avatar Dec 12 '25 12:12 github-actions[bot]

Backport

To backport this PR, click the button next to the target branch and then click "Run workflow" in the Run Actions UI.

Branch Run
stable-25-2, stable-25-2-1, stable-25-3, stable-25-3-1 ▶  Backport
stable-25-3, stable-25-3-1 ▶  Backport
stable-25-3 ▶  Backport

▶  Backport manual

ydbot avatar Dec 12 '25 15:12 ydbot