Additional storage parameters

[kvaps]

From here https://github.com/aenix-io/etcd-operator/pull/67#discussion_r1535868169

The new spec looks good good. The only question if we do this:

storage:
   volumeClaimTemplate:
     metadata:
       labels:
         env:prod
       annotations:
         example.com/annotation: "true"
     spec: # core.v1.PersistentVolumeClaimSpec Ready k8s type
       storageClassName: gp3
       accessModes: ["ReadWriteOnce"]
       resources:
         requests:
           storage: 10Gi
   emptyDir: {} # core.v1.EmptyDirVolumeSource Ready k8s type

then where will we add the possible options in the future:

1. auto-deletion of PVC after deleting the EtcdCluster object
2. options for resizing PVC (resizeInUseVolumes as CNPG and mariadb-operator do
3. option to manage storage migration
4. any other options for storage in the future

[kvaps]

I'm thinking about additional level, something like this:

storage:
  resizeInUseVolumes: true
  additionalOptionHere: true
  volume: # OneOf
    emptyDir: {}
    volumeClaimTemplate: {}

What is your ideas?

[AlexGluck]

Point 1 and 2 agree, but in point 3 how was work storage migration logic? Before anyone give suggestions we don't must change spec. And for point 4 it is premature optimization.

[kvaps]

storage:
  resizeInUseVolumes: true
  additionalOptionHere: true
  emptyDir: {}
  volumeClaimTemplate: {}