sentry icon indicating copy to clipboard operation
sentry copied to clipboard

fix(deletions): Fix BulkDeleteQuery pagination

Open yuvmen opened this issue 1 month ago • 2 comments

BulkDeleteQuery uses RangeQuerySetWrapper to iterate over results use a non-unique fields - last_seen. This actually doesnt work and skips values on rows with same last_seen. The only way to solve this and paginate correctly is to paginate over a complex combined condition of both last_seen and id. To do this RangeQuerySetWrapper has been adapted to support a list to order_by, expecting this list to end with a unique key. Added tests and converted BulkDeleteQuery to use it.

yuvmen avatar Dec 05 '25 00:12 yuvmen

Codecov Report

:x: Patch coverage is 98.73418% with 1 line in your changes missing coverage. Please review. :white_check_mark: All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/sentry/db/deletion.py 90.90% 1 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##           master   #104435       +/-   ##
============================================
+ Coverage   63.54%    80.55%   +17.00%     
============================================
  Files        9347      9348        +1     
  Lines      399913    400452      +539     
  Branches    25646     25615       -31     
============================================
+ Hits       254128    322587    +68459     
+ Misses     142986     77417    -65569     
+ Partials     2799       448     -2351     

codecov[bot] avatar Dec 05 '25 00:12 codecov[bot]

@wedamija alright refactored quite a bit - still left result_value_getter as not always returning a dict since that would force changing every call to RangeQuerySetWrapper across the entire codebase which felt like too big of a disruption, and also very specific since I need every place to know to write {"pk": item} which is cumbersome.

yuvmen avatar Dec 08 '25 22:12 yuvmen