quickwit
quickwit copied to clipboard
Implement `reset_index_checkpoint` and `reset_source_checkpoint` metastore APIs and CLI commands
Description
Per title.
How was this PR tested?
- added unit test
- ran
make test-all
command
As discussed offline with @fulmicoton, we currently don't have a pressing need for this API. We will revisit when necessary.
Reopened. See #1631.
This PR is ready for review.
I have introduced two new metastore APIs, which completely clears the checkpoints (keys are removed)
-
reset_index_checkpoint(index_id)
-
reset_source_checkpoint(index_id, source_id)
I have introduced two new CLI commands:
-
index reset-checkpoint --index <index ID>
-
source reset-checkpoint --index <index ID> --source <source ID>
PTAL!
FYI, in relation to this as part of the Kafka offset storage I've been working on metastore level changes to move checkpoints into a separate table on Postgres backed metastores. As part of that I'm adding two APIs, one of which would overlap with this.
/// Retrieves the current checkpoints for an index
///
/// Returns a `IndexCheckpoint` for the provided `index_id` and optionally filtered by
/// `source_id`, `resource` and `group_id`.
/// If no checkpoint has been stored yet the result will be an empty `IndexCheckpoint`.
async fn index_checkpoint(
&self,
index_id: &str,
source_id: Option<String>,
resource: Option<String>,
group_id: Option<String>,
) -> MetastoreResult<IndexCheckpoint>;
/// Removes the current checkpoints for an index
///
/// Removes all checkpoints that match the provided `index_id` and optionally filtered by
/// `source_id`, `resource` and `group_id`.
/// If no checkpoint has been stored it will not be considered an error.
async fn delete_index_checkpoint(
&self,
index_id: &str,
source_id: Option<String>,
resource: Option<String>,
group_id: Option<String>,
) -> MetastoreResult<()>;