pulumi-kubernetes
pulumi-kubernetes copied to clipboard
Helm Release fails if Chart uses .Capabilities.APIVersions
What happened?
I was trying to install Cilium's Helm Chart using Pulumi and once I enabled the ServiceMonitor option it failed, even though the cluster I was targeting had prometheus-operator CRDs installed in it.
When I investigated the chart I found it was using .Capabilities.APIVersions.Has
to force it to error out if the CRD wasn't installed. The lines in the Helm file which are causing this issue are specified here https://github.com/cilium/cilium/blob/master/install/kubernetes/cilium/templates/validate.yaml#L18-L23.
I did find a closed issue in Cilium's repo which asked for a way to override this check but they closed the ticket as depending on the Kubernetes version and API versions available it changes what the chart outputs https://github.com/cilium/cilium/issues/16552
When I looked through Pulumi's code I saw that APIVersions is set to nil when passed into Helm for rendering https://github.com/pulumi/pulumi-kubernetes/blob/5277945f4d5fd63032d3f7014e27fb113335eeaa/provider/pkg/provider/helm_release.go#L1107
Therefore, I need a way to either manually specify what the APIVersions are or that Pulumi automatically detects the APIVersions installed in the target cluster and passes it into Helm.
Steps to reproduce
Created an empty stack and ensured that kubectl is pointing at a cluster with Prometheus Operator CRDs installed in it
import * as pulumi from '@pulumi/pulumi'
import * as k8s from '@pulumi/kubernetes'
const cilium = new k8s.helm.v3.Release('cilium', {
chart: 'cilium',
version: '1.12.0',
repositoryOpts: {
repo: 'https://helm.cilium.io/'
},
namespace: 'kube-system',
values: {
prometheus: {
enabled: true,
serviceMonitor: {
enabled: true
}
}
}
})
Expected Behavior
The helm release is deployed without error into the cluster which already contains the prometheus-operator CRD's
Actual Behavior
❯ pulumi preview
Previewing update (archie-test):
Type Name Plan Info
+ pulumi:pulumi:Stack archie-test-archie-test create
└─ kubernetes:helm.sh/v3:Release cilium 1 error
Diagnostics:
kubernetes:helm.sh/v3:Release (cilium):
error: failed to create chart from template: execution error at (cilium/templates/validate.yaml:21:9): Service Monitor requires monitoring.coreos.com/v1 CRDs. Please refer to https://github.com/prometheus-operator/prometheus-operator/blob/master/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml
Versions used
❯ pulumi about
CLI
Version 3.37.2
Go Version go1.18.4
Go Compiler gc
Plugins
NAME VERSION
kubernetes 3.20.4
nodejs unknown
Host
OS darwin
Version 12.5
Arch x86_64
This project is written in nodejs: executable='/Users/archie/.nvm/versions/node/v16.16.0/bin/node' version='v16.16.0'
Backend
Name ARCHIE-LAPTOP
URL gs://pulumi-state
User archie
Organizations
Dependencies:
NAME VERSION
@pulumi/pulumi 3.37.2
@types/node 14.18.24
@pulumi/kubernetes 3.20.4
Pulumi locates its logs in /var/folders/7f/4wh6789d7bl12zbm5kqf65m00000gp/T/ by default
warning: Failed to get information about the current stack: No current stack
Additional context
I know the CRDs are installed correctly because kubectl reported they are created
❯ kubectl get customresourcedefinitions.apiextensions.k8s.io servicemonitors.monitoring.coreos.com
NAME CREATED AT
servicemonitors.monitoring.coreos.com 2022-06-16T15:52:25Z
Contributing
Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).