temporalio
[Feature Request] Add support for deploying and using PostgreSQL
Is your feature request related to a problem? Please describe.
While it is possible to use an external PostgreSQL database, the Helm Chart provides no support for deploying and configuring one within a Kubernetes cluster, as it does for Apache Cassandra. This issue requests that support to be added (similar to what is requested for MySQL in #635).
Describe the solution you'd like
A cluster deployed through the Helm Chart would continue to use Cassandra and Elasticsearch by default, but one could optionally set a property postgresql.enabled to true to deploy and use a PostgreSQL container for both persistence and visibility instead. There may be additional properties for customizing aspects, such as the username and password, for that PostgreSQL deployment.
This issue does not specifically request the ability to use PostgreSQL for persistence and another SQL system (such as MySQL) for visibility. The only expected supported combinations are PostgreSQL for both, and PostgreSQL for persistence with Elasticsearch for visibility. Support for other combinations may be added in the future, but need not be addressed for the initial implementation described here.
Helm best practises should be followed so that choosing to use a sub-chart or an external system are done in a predictable and clear way. Where possible we should avoid the need to duplicate values, the system should infer the Postgres hostname from the sub-chart, and so on.
Additional context
The PostgreSQL version deployed should be the latest stable version supported by the Temporal Server software being deployed. In the case of known incompatibilities with that version of PostgreSQL, it might deploy a different version.
I've just recently gone through this exercise using CrunchyData's postgresql operator and the biggest issue I've run into is that it's very hard to actually provide a custom user and password that isn't defined strictly into the helm chart. In this operator's mechanization, it creates a user name and password stored in a secret within K8s. Providing these secrets into the chart is very non-trivial.
the entire open source journey is doubtful by temporal. external postgres is not supported properly, and there is 0 managed Cassandra after DataStax got acquired by ibm, you will not bet on their future, simply because it's unclear how big company will treat this db product.
I am facing similar the same challenge as well while setting up temporal using helm. I want to just use postgres, grafana, and prometheus. I do not want to use cassandra or elasticsearch to avoid management overhead. For testing, I have disabled prometheus and grafana as well using command helm install -f values/values.postgresql.yaml --set cassandra.enabled=false --set elasticsearch.enabled=false --set prometheus.enabled=false --set grafana.enabled=false temporaltest temporal --timeout 900s. However, I still see that pods for temporal-cassandra, elasticsearch, etc. are getting generated. This feature request is valid.
@lindleywhite We support existingSecret already for Postgres, if that is not working then please file an issue.
@sneerin Please file issues for anything preventing you from using external Postgres. External Postgres and MySQL are widely used, particularly against RDS/Aurora, so I'm not sure what is missing for you.
Neither of these comments are related to supporting installing Postgres via subchart though, so please create issues as appropriate.
existingSecret does work for persistence passwords, but it doesn't enable the ability to dynamically provide non-static user.
That's fine I guess, but if you want to provision DB users dynamically for Temporal, you couldn't. I worked around this by using a static user temporal for every database. But if I wanted a shared DB cluster between N of clusters, they would need to use the same user, which is not ideal.
We should definitely support reading a username from a secret also. I'm happy to add that to the list of v1 features, @lindleywhite please file an issue.
We should definitely support reading a username from a secret also. I'm happy to add that to the list of v1 features, @lindleywhite please file an issue.
I would prefer my secret containing the entire Postgres connection string like how it is done in the Prometheus Postgres exporter: https://github.com/prometheus-community/helm-charts/blob/7e8cc15d1106e55b91438610f223aa762c201be3/charts/prometheus-postgres-exporter/values.yaml#L183
Docs provide instructions about how to use "your own postgresql". If this actually does not work AT ALL, please don't leave misleading documentation like that in the main branch. It wasted a lot of my time.
