spqr icon indicating copy to clipboard operation
spqr copied to clipboard

Lack of diagnostics during `REDISTRIBUTE KEY RANGE`

Open Denchick opened this issue 2 months ago • 0 comments

Currently, there is no clear way to track the progress or internal state of data transfer:

spqr-console=> REDISTRIBUTE KEY RANGE kr3 TO sh2 BATCH SIZE 1000 CHECK;
    redistribute key range
------------------------------
 key range id         -> kr3
 destination shard id -> sh2
 batch size           -> 1000
(3 rows)

spqr-console=> REDISTRIBUTE KEY RANGE kr3 TO sh2 BATCH SIZE 1000;

It’s unclear how the redistribution actually proceeds. At the moment, the only available way to observe progress are:

  • manually checking the etcd path /current_task_index
  • manually checking the etcd path "/move_tasks/fab40bad-7446-422a-97e0-ec5cd83bfa93"
  • SHOW task_group (no idea whether this command support filtering)

Proposed ideas:

  • Add a SHOW REDISTRIBUTION STATUS or DESCRIBE TASK command in spqr-console
  • Expose additional diagnostic data in etcd (e.g., progress percentage, current stage, or active transactions)
  • Log redistribution steps at the coordinator level for better observability

Denchick avatar Oct 22 '25 11:10 Denchick