spark-operator
spark-operator copied to clipboard
[wip] Stateful sets
Changing what operator actually deploys. Replication controllers are considered as a low-lvl detail k8s resources and don't allow for durable data (one can't assign the persistent volume to a certain replica managed by RC).
This changes replication controllers to stateful sets, it's wip, because the volumeClaimTemplates are still not there.
Description
after applying the example/cluster.yaml:
λ oc get rc
No resources found.
λ oc get statefulsets
NAME READY AGE
my-spark-cluster-m 1/1 3m6s
my-spark-cluster-w 2/2 3m6s
:sparkles: New feature (non-breaking change which adds functionality)
transform the replication controllers into stateful sets if the PV is asked
https://github.com/radanalyticsio/spark-operator/blob/fc2d038531213e0813f9aa955f8b978bb8359e93/src/main/java/io/radanalytics/operator/cluster/KubernetesSparkClusterDeployer.java#L212-L217