druid icon indicating copy to clipboard operation
druid copied to clipboard

Refactor `SegmentLoadDropHandler` code

Open abhishekrb19 opened this issue 1 year ago • 0 comments

Marking this as draft to let CI run and a few more things need to be addressed.

The SegmentLoadDropHandler code currently supports the following functionalities:

  1. Bootstrapping on startup: a. Loading previously cached segments b. Fetching and loading bootstrap segments from the coordinator
  2. Loading segments
  3. Dropping segments
  4. Batch operations

The current structure of the code makes it difficult to add new functionality. Therefore, this PR proposes splitting the functionality into two separate components:

  1. SegmentBootstrapper: Responsible for bootstrapping functions
  2. SegmentLoadDropHandler: Responsible for segment change handling functions — addSegment(), removeSegment(), and processBatch()

Apart from improving code hygiene, this refactoring allows for customizations in the bootstrapping code without affecting the load/drop functionality. Some potential customizations in the bootstrapping flow include selective loading of segments by tasks and server-specific segment loading.

This change does not introduce any new functional changes.

This PR has:

  • [ ] been self-reviewed.
    • [ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
  • [ ] added documentation for new or modified features or behaviors.
  • [ ] a release note entry in the PR description.
  • [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • [ ] added or updated version, license, or notice information in licenses.yaml
  • [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • [ ] added integration tests.
  • [ ] been tested in a test Druid cluster.

abhishekrb19 avatar Jul 02 '24 18:07 abhishekrb19