Slack tokenSecretRef ignored in ContactPoint

[excalq]

Crossplane Version

v1.16.0

Crossplane Grafana Provider Version

v0.22.0

Affected Resource(s)

• alerting.grafana.crossplane.io/v1alpha1/ContactPoint

(May affect other resources with tokenSecretRefs?)

YAML resources

apiVersion: alerting.grafana.crossplane.io/v1alpha1
kind: 
metadata:
  name: grafana-alerts-contactpoint-slack--devops-alerts-testing
spec:
  providerConfigRef:
    name: provider-grafana
  forProvider:
    name: "Crossplane IaC: Slack: #devops-alerts-testing"
    slack:
      - recipient: devops-alerts-testing
        tokenSecretRef: 
          key: grafana-external-secrets
          name: slackTokenGrafanaAlerting
          namespace: crossplane-system

apiVersion: v1
kind: Secret
metadata:
  name: grafana-external-secrets
  namespace: crossplane-system
data:
  grafana-provider-auth: <redacted_api_token>
  slackTokenGrafanaAlerting: <redacted_slack_xoxb_token>
type: Opaque

Expected Behavior

The Grafana Crossplane provider should POST to the Grafana API, having a payload which includes the token.

In Terraform, this token is a simple (required) string under the slack schema.

In Crossplane, the token is sourced from a K8s secret, specified by tokenSecretRef.

Actual Behavior

The API post is rejected as invalid due to the token being missing:

Warning CannotCreateExternalResource 1s (x15 over 8m19s) managed/alerting.grafana.crossplane.io/v1alpha1, kind=contactpoint failed to create the resource: [{0 [POST /v1/provisioning/contact-points][400] postContactpointsBadRequest {"message":"invalid object specification: failed to validate integration \"Crossplane IaC: Slack: #devops-alerts-testing\" (UID ) of type \"slack\": token must be specified when using the Slack chat API"} []}]

Upon configuring the Grafana provider to run with a debug configuration, the debug logs include this clue, suggesting the token is completely missing/blank:

\"slack.1515124646.token\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:true, Type:0x0},

Excepted from the full logs:

2024-12-19T19:36:16Z	DEBUG	provider-grafana	Diff detected	{"uid": "1b786dea-e1a6-4b28-8cef-730f08126f5d", "name": "grafana-alerts-contactpoint-slack--devops-alerts-testing", "gvk": "alerting.grafana.crossplane.io/v1alpha1, Kind=ContactPoint", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"disable_provenance\":*terraform.ResourceAttrDiff{Old:\"\", New:\"false\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, \"name\":*terraform.ResourceAttrDiff{Old:\"\", New:\"Crossplane IaC: Slack: #devops-alerts-testing\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, \"slack.#\":*terraform.ResourceAttrDiff{Old:\"0\", New:\"1\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.disable_resolve_message\":*terraform.ResourceAttrDiff{Old:\"\", New:\"false\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.endpoint_url\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.icon_emoji\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.icon_url\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.mention_channel\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.mention_groups\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.mention_users\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.recipient\":*terraform.ResourceAttrDiff{Old:\"\", New:\"devops-alerts-testing\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.text\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.title\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.token\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:true, Type:0x0}, \"slack.1515124646.uid\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"slack.1515124646.url\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:true, Type:0x0}, \"slack.1515124646.username\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}
2024-12-19T19:36:16Z	DEBUG	provider-grafana	Creating the external resource	{"uid": "1b786dea-e1a6-4b28-8cef-730f08126f5d", "name": "grafana-alerts-contactpoint-slack--devops-alerts-testing", "gvk": "alerting.grafana.crossplane.io/v1alpha1, Kind=ContactPoint"}
2024-12-19T19:36:16Z	DEBUG	provider-grafana	Cannot create external resource	{"controller": "managed/alerting.grafana.crossplane.io/v1alpha1, kind=contactpoint", "request": {"name":"grafana-alerts-contactpoint-slack--devops-alerts-testing"}, "uid": "1b786dea-e1a6-4b28-8cef-730f08126f5d", "version": "173182014", "external-name": "", "error": "failed to create the resource: [{0 [POST /v1/provisioning/contact-points][400] postContactpointsBadRequest {\"message\":\"invalid object specification: failed to validate integration \\\"Crossplane IaC: Slack: #devops-alerts-testing\\\" (UID ) of type \\\"slack\\\": token must be specified when using the Slack chat API\"}  []}]", "errorVerbose": "failed to create the resource: [{0 [POST /v1/provisioning/contact-points][400] postContactpointsBadRequest {\"message\":\"invalid object specification: failed to validate integration \\\"Crossplane IaC: Slack: #devops-alerts-testing\\\" (UID ) of type \\\"slack\\\": token must be specified when using the Slack chat API\"}  []}]\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKExternal).Create\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_tfpluginsdk.go:624\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/reconciler/managed/reconciler.go:1058\ngithub.com/crossplane/crossplane-runtime/pkg/ratelimiter.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/ratelimiter/reconciler.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1223"}
2024-12-19T19:36:16Z	DEBUG	provider-grafana.events	failed to create the resource: [{0 [POST /v1/provisioning/contact-points][400] postContactpointsBadRequest {"message":"invalid object specification: failed to validate integration \"Crossplane IaC: Slack: #devops-alerts-testing\" (UID ) of type \"slack\": token must be specified when using the Slack chat API"}  []}]	{"type": "Warning", "object": {"kind":"ContactPoint","name":"grafana-alerts-contactpoint-slack--devops-alerts-testing","uid":"1b786dea-e1a6-4b28-8cef-730f08126f5d","apiVersion":"alerting.grafana.crossplane.io/v1alpha1","resourceVersion":"173182433"}, "reason": "CannotCreateExternalResource"}

Steps to Reproduce

1. Provision Grafana with a Service Account + Token (having admin rights)
2. Provision Crossplane using this provider, configured to auth with the SA-Token
3. Attempt to provision the above ContactPoint, using the established ProviderConfig.

Important Factoids

Grafana is running as an OSS install (helm installed), in the same Kubernetes cluster as Crossplane, but in a different namespace. The Grafana API url is http://grafana.monitoring.svc.cluster.local in the ProviderConfig.

I did duplicate the secret containing the slack token to the crossplane-system namespace, but that had change in the condition versus using the monitoring namespace.

A brief search in the Terraform provider issues show no related issues.

References

No response

[Duologic]

The resource ref has:

          key: grafana-external-secrets
          name: slackTokenGrafanaAlerting

However the secret has:

  name: grafana-external-secrets
data:
  slackTokenGrafanaAlerting: <redacted_slack_xoxb_token>

It looks like you switched the name/key here.

[excalq]

Thanks a ton, that was the issue indeed. I'll close this issue.