Passing spec configuration to etcd operator

[danilina-wsib]

Hi,

How can I define custom options for the etcd cluster. Specifically, etcd operator supports persistent volume claims, beside other stuff I might want to customize. Is there any way to pass this config to etcd operator?

Thank you, Alexei Daniline

[rblaine95]

It looks like this function creates the etcd cluster with a specific definition.

Modifying it slightly would allow, for example, PVCs to be defined as well.

Edit: The problem with this solution is that it involves editing the source code of the vault operator. You can't do that in a compiled Docker running in kubernetes. So, you'd have to fork the project and build your own custom docker and configure that with your custom code.

Edit 2: I was being an absolute idiot with my original code. Here is code that should work (pending testing)

// vault-operator/pkg/apisvailt/v1alpha1/types.go
type VaultServiceSpec struct {
	// Number of nodes to deploy for a Vault deployment.
	// Default: 1.
	Nodes int32 `json:"nodes,omitempty"`

	// Base image to use for a Vault deployment.
	BaseImage string `json:"baseImage"`

	// Version of Vault to be deployed.
	Version string `json:"version"`

	// Pod defines the policy for pods owned by vault operator.
	// This field cannot be updated once the CR is created.
	Pod *PodPolicy `json:"pod,omitempty"`

	// Name of the ConfigMap for Vault's configuration
	// If this is empty, operator will create a default config for Vault.
	// If this is not empty, operator will create a new config overwriting
	// the "storage", "listener" sections in orignal config.
	ConfigMapName string `json:"configMapName"`

	// TLS policy of vault nodes
	TLS *TLSPolicy `json:"TLS,omitempty"`

	// PersistentVolumeClaimSpec for the ETCD Cluster
	PersistentVolumeClaimSpec *v1.PersistentVolumeClaimSpec `json:"PersistentVolumeClaimSpec,omitempty"`
}

// vault-operator/pkg/util/k8sutil/vault.go
func DeployEtcdCluster(etcdCRCli etcdCRClient.Interface, v *api.VaultService) error {
	size := 3
	etcdCluster := &etcdCRAPI.EtcdCluster{ // Deploy the etcd Cluster with the PVC spec
		TypeMeta: metav1.TypeMeta{
			Kind:       etcdCRAPI.EtcdClusterResourceKind,
			APIVersion: etcdCRAPI.SchemeGroupVersion.String(),
		},
		ObjectMeta: metav1.ObjectMeta{
			Name:      EtcdNameForVault(v.Name),
			Namespace: v.Namespace,
			Labels:    LabelsForVault(v.Name),
		},
		Spec: etcdCRAPI.ClusterSpec{
			Size: size,
			TLS: &etcdCRAPI.TLSPolicy{
				Static: &etcdCRAPI.StaticTLS{
					Member: &etcdCRAPI.MemberSecret{
						PeerSecret:   EtcdPeerTLSSecretName(v.Name),
						ServerSecret: EtcdServerTLSSecretName(v.Name),
					},
					OperatorSecret: EtcdClientTLSSecretName(v.Name),
				},
			},
			Pod: &etcdCRAPI.PodPolicy{
				EtcdEnv: []v1.EnvVar{{
					Name:  "ETCD_AUTO_COMPACTION_RETENTION",
					Value: "1",
				}},
				PersistentVolumeClaimSpec: v.Spec.PersistentVolumeClaimSpec,
			},
		},
	}
//snip

Edit 3: This implementation makes a PersistentVolumeClaimSpec a requirement in the vault manifest. I've got a fork here with custom service account for vault in the master branch (currently in a PR into this repository #324) and a branch for the etcd pvc stuff. Will probably do some testing on my side and then open a PR for this.

[rblaine95]

If you want to run the changes that I've made, head over to our Quay repository.

The tag 0.1.9-1 is a build of the latest stable vault operator release with my changes (custom service account and PVC for ETCD) included.

# Deploy vault operator onto OpenShift
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: vault-operator
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: vault-operator
    spec:
      serviceAccountName: vault-operator
      containers:
      - name: vault-operator
        # image: quay.io/coreos/vault-operator:latest
        image: quay.io/zenlab/vault-operator:0.1.9-1
        env:
        - name: MY_POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name

# Deploy vault service onto OpenShift
apiVersion: vault.security.coreos.com/v1alpha1
kind: VaultService
metadata:
  name: vault
spec:
  nodes: 3
  version: 0.9.1-1
  serviceAccountName: vault-sa # use vault-sa as the service account
  persistentVolumeClaimSpec: # add a persistent volume claim to etcd
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 5Gi

[drspockbr]

Any chance to have this feature merged to the main code of vault-operator? I'm facing the same issue here and having that would save lot of time.

[rblaine95]

@drspockbr #332 banzaicloud/bank-vaults