cf-ops-automation
cf-ops-automation copied to clipboard
add k8s support
As cloudfoundry now supports Kubernetes with CloudFoundry Container Runtime, COA should offer first class support
- [x] add k8s terraform plugin, so we can apply k8s config from terraform
- [x] add helm chart support via terraform plugin
- [ ] enable k8s containers provisionning (eg k8s-apps-depls naming convention, use a declarative format like helm, add kubctl to the pre / post scripts hosting containers)
cc @obeyler
Including helm chart requires investigations, 2 providers exist:
- https://github.com/mcuadros/terraform-provider-helm with some drawbacks preventing it to be an official one: https://github.com/hashicorp/terraform/issues/14126
- https://github.com/djhaskin987/terraform-provider-helmcmd: addtionnal requirements (kubectl and helm binaries) and not yet used by the community
mcuadros provider seem to have the biggest user community support. Seems the way to go short term
Helm is now part of the base image: https://github.com/orange-cloudfoundry/paas-docker-cloudfoundry-tools/commit/ad445d6b34dffeadb3c2b26a40dd71de73ec0686. But not yet used by COA.
Base image is now used by COA (from https://github.com/orange-cloudfoundry/cf-ops-automation/commit/27e680550bae917c2e3d8e613b3033a39ce48162) and will be part of next release (2.0.0)
Merging #180 into this issue
Requirements:
- install helm chart
- including custom params
- validate/test the chart by running associated tests if any, see chart_tests.md for background
The considered Tf provider is https://github.com/mcuadros/terraform-provider-helm
Steps:
- install the provider
- set up the provider prereq in the docker image: kubectl +
helm init
(as a workaround for https://github.com/mcuadros/terraform-provider-helm/issues/23)
Remaining issues:
- the chart repository needs to be versioned and fresh. The concourse workers are stateless. When using the helm_repository resource, the set up repository is ephemeral. The format of the chart repository and its content is documented into https://github.com/helm/helm/blob/master/docs/chart_repository.md which complements charts detailed structure at https://github.com/helm/helm/blob/master/docs/charts.md
Possible workarounds:
- have the provider support a new
chart_repository
data source, so that the chart repository is donwloaded systematically - have COA persist the chart repository directory
- into the paas-secret git repo
- by syncing the container with an external store prior and after the TF plan/apply command
- don't use the helm_repository resource and rather execute helm commands prior to the TF execution (that will only apply the "helm_release" resource). TBD how
/CC @poblin-orange @obeyler
Draft issue on https://github.com/mcuadros/terraform-provider-helm/ to suggest datasource.
@obeyler we need to record traces and behavior of the provider terraform-provider-helm when the helm_repository
files are reported as missing
Issue title: Add support for a helm_repository datasource
### Actual Behavior
Currently the [helm_repository](https://github.com/mcuadros/terraform-provider-helm/blob/master/docs/repository.md) resources downloads the repository on local disk in `$HELM_HOME` and expects it to be persistent across TF executions
When this provider is used within stateless continuous integration systems, the local disk is ephemeral (along with any downloaded repository), so the files associated to the `helm_repository` are missing at each Tf plan/apply execution, and gets recreated
In addition, plain CI systems need the local environment setup as described in #23
### Expected Behavior
Similarly to the [aws_s3_bucket_object](https://www.terraform.io/docs/providers/aws/d/s3_bucket_object.html) which provides a way to download a S3 bucket content in memory and use it from other resource, a `helm_repository` data source would systematically download the repository locally (in memory or local ephemeral disk).
This way stateless CI/CD systems can merely add static helm configuration, and don't need persist changes make to $HELM_HOME by the `terraform-provider-helm`