GreptimeTeam
Storage engine should support delete
What problem does the new feature solve?
Now our storage engine only supports put but doesn't allow deleting a row.
What does the feature do?
Implements a delete() API for the storage engine. Like regular LSM-Tree-based engines, we could add a Delete variant to OpType, then when we find a row with OpType::Delete, then we consider that row has been deleted.
Implementation challenges
We could only support deleting a row by its row key (and timestamp) first, which is the simplest case and should be easy to implement.
GreptimeDB is a time sequence database, so we can't add delete tag to overwrite the original value like LSM, we can only modify optype.delete in place through key and timestamp, if so, it feels like random IO will happen. One is the impact of performance, and the second is the timing of the deletion scene, I feel that almost no scene need to delete.
We also use the storage engine to store things like the system catalog table, which contains the catalog's metadata. It would be useful to support deleting a metadata entry. Users could also use this feature to revise their data.
so we can't add delete tag to overwrite the original value like LSM
We only need to support deleting a specific point (row), so the implementation is exactly the same as a regular LSM engine.
I see your idea, can I have a try？
Oops, I forgot to self assign this issue, I'm working on this feature.