docs
docs copied to clipboard
cockroachdb
Investigate SSD recommendations and whether GKE / Kubernetes info is outdated
Kevin Kokomani (kevinkokomani) commented:
Docs in question: https://www.cockroachlabs.com/docs/v20.2/recommended-production-settings.html#gcp & https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html#local-disks.
Recently we recommended a user use local SSDs on GCP according to the production checklist, which implies that we fully support it as a feature: https://www.cockroachlabs.com/docs/v20.2/recommended-production-settings.html#gcp.
Use Local SSDs or SSD persistent disks. Note that the IOPS of SSD persistent disks depends both on the disk size and number of vCPUs on the machine.
The user pointed out that they're using kubernetes, and the documentation they were following mentions local PersistentVolumes being still in beta support in k8s v1.10: https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html#local-disks
However, using local disks typically provides better performance than remotely attached disks, such as SSD Instance Store Volumes instead of EBS Volumes on AWS or Local SSDs instead of Persistent Disks on GCE. StatefulSets have historically not supported using local disks, but beta support for using "local" PersistentVolumes was added in Kubernetes v1.10. We do not recommend using this for production data until the feature is more mature, but it's a promising development.
The user is wondering if Cockroach now thinks that local SSDs are a production-ready solution given that our k8s specific docs say otherwise.
Ask: I believe our docs are outdated, since Kubernetes announced local persistent volumes being in general availability as of v1.14: https://kubernetes.io/blog/2019/04/04/kubernetes-1.14-local-persistent-volumes-ga/. I believe the solution here is to confirm with the product team that since local SSDs are no longer considered beta for Kubernetes, we recommend it as a production-ready feature and update our k8s docs accordingly: https://www.cockroachlabs.com/docs/stable/kubernetes-performance.html#local-disks. If it's still not recommended, we may want to update our production checklist docs (https://www.cockroachlabs.com/docs/v20.2/recommended-production-settings.html#gcp) to have the caveat if you're using Kubernetes.
Jira Issue: DOC-1096
To add to this - we offer a suggested stateful set configuration YAML file that's linked out from here: https://www.cockroachlabs.com/docs/v20.2/orchestrate-a-local-cluster-with-kubernetes-insecure.html?filters=manual#step-2-start-cockroachdb. This particular one does not use local SSDs, so if we do support this configuration in production, the product team might want to consider creating one for this kind fo configuration.
