glusterd2
glusterd2 copied to clipboard
gluster
Implement Snapshot Config
Labels are used to tag different object, and perform collective operation based on label configuration values
More information on issue #1094
Signed-off-by: Mohammed Rafi KC [email protected]
As discussed offline, I expected some discussion on the issue first. There's clearly a overlap between metadata and labels. See these issues for some previous discussions:
https://github.com/gluster/glusterd2/issues/462 https://github.com/gluster/glusterd2/issues/1018 https://github.com/gluster/glusterd2/issues/1018
@prashanthpai , As far as I understand Metadata can only use as a tag. It cannot have different states, or in other words, it can not perform collective actions of its own. Using metadata, We can list all snapshots, delete all snapshots. But it cannot take intelligent decisions like when should we delete.
For that reason, extending metadata won't be a feasible solution at the moment. Thoughts ?
"tags" should be key=value pairs. While they aren't to be interpreted by "core", it's fair game for plugins and peripheral entities to use them. For example, the operator will communicate w/ IVP and auto volume balancing (I need a new name) via the value of node-level tags.
Is key=value sufficient for your purposes?
@rafikc30 What is generally referred as tags or labels in other projects, we call them metadata (key-value pairs). They are not meant to be interpreted by core. There is no inherent meaning (or actions) attached to these arbitrary strings in the core glusterd2.
If you wish to implement snapshot scheduler as an external entity/daemon or a glusterd2 plugin, you can use the metadata infrastructure.
But it cannot take intelligent decisions like when should we delete
The core is not supposed to - regardless of whether you call them labels, tags or metadata.
On the issue you mention:
The first consumer of labels will be Snapshot. We can use labels to list snapshot, delete the snapshot and perform other high-level actions.
You can use metadata for all of these usecases. These higher level actions are performed by user or some external entity. For example:
- list all snapshots that have has a label/tag
backup - delete all snapshots that have a label/tag
thumbnails - delete all snapshots that have
timestamplabel/tag set to a value older than certain date
For example, In a customer setup, a snapshot can schedule based on different labels like Monthly, Hourly, Yearly, Daily etc. Each one of this can have different configuration values. In a typical set up, the maximum number of a snapshot for the Hourly label can set to 24, while the same value will be 12 for the Monthly snapshot. So performing auto-delete based on this max values will be based on label values.
The core of glusterd2 will not be doing these automatic operations. You can do it as an external daemon or a plugin.
As snapshot is considered core in glusterd2, if you want these automatic operations to be done as core, you might as well make these operations concrete with a well-defined API to schedule+prune snapshots and not use metadata which is arbitrary.
What other specific and real-world usecase does your labels proposal solve beyond snapshot for it to be a generic infrastructure ?
As pointed out earlier, this discussion is best done over the issue.
@rafikc30 was there any further discussion about metadata vs labels between you and @prashanthpai? The conversation seems to have stopped in between.
The discussion were moved to the issue https://github.com/gluster/glusterd2/issues/1094. Initially the PR was submitted as labels being a general use case api where it can be used for volumes,snapshots, etc. But then we decided to move to snapshot alone (please see the issue 1094), because at the moment snapshot were the only consumer.