accumulo icon indicating copy to clipboard operation
accumulo copied to clipboard

Add compaction IT that verifies queues are cleared when tablets no longer need to compact.

Open keith-turner opened this issue 11 months ago • 2 comments

This is follow on work for #3528 and #4394. Would be good to add an integration test for the changes in #4394. Unit test were added in #4394 that cover a narrow part of the code, but it would be good to verify things work end to end. Should be able to do something like the following in an IT.

  • Create a table w/ N tablets
  • Configure the table to use compaction group G1
  • Start no compactors processes for G1. This should cause compactions to queue but not run.
  • Add files and change compaction settings such that all tablets need to compact, could lower the compaction ratio and/or add more files per tablet than the compaction ratio
  • Verify N tablets are queued for G1 via inspecting metrics
  • Change the table config in such a way that no tablets need to compact, could raise the compaction ratio. Once the tablets no longer need to compact they should be cleared from the queue.
  • Verify zero tablets are queued for G1 via inspecting metrics

keith-turner avatar Mar 18 '24 17:03 keith-turner

It may be worthwhile to add this test to 2.1 and forward. The implementations are very different, but for this case hopefully the behavior is the same.

keith-turner avatar Mar 18 '24 18:03 keith-turner

I can start looking into this.

DomGarguilo avatar Mar 18 '24 20:03 DomGarguilo

Closed via #4466

DomGarguilo avatar May 17 '24 19:05 DomGarguilo