redpanda-data
5.92: connector error in console.cleanForK8s
What happened?
Logs show :
{"level":"error","ts":"2024-09-03T21:55:18.770Z","msg":"Reconciler error","controller":"helmrelease","controllerGroup":"helm.toolkit.fluxcd.io","controllerKind":"HelmRelease","HelmRelease":{"name":"redpanda","namespace":"buggy"},"namespace":"buggy","name":"redpanda","reconcileID":"3cb93fba-fc84-4f8e-afbf-be5866d7c12a","error":"template: redpanda/templates/tests/test-connector-via-console.yaml:150:24: executing \"redpanda/templates/tests/test-connector-via-console.yaml\" at <include \"console.fullname\" $consoleValues>: error calling include: template: redpanda/charts/console/templates/_helpers.tpl:16:10: executing \"console.fullname\" at <include \"console.Fullname\" (dict \"a\" (list .))>: error calling include: template: redpanda/charts/console/templates/_helpers.go.tpl:22:30: executing \"console.Fullname\" at <include \"console.cleanForK8s\" (dict \"a\" (list $values.fullnameOverride))>: error calling include: template: redpanda/charts/console/templates/_helpers.go.tpl:78:48: executing \"console.cleanForK8s\" at <$s>: invalid value; expected string"}
What did you expect to happen?
It to not fail
How can we reproduce it (as minimally and precisely as possible)?. Please include values file.
repro
follow install guide for helm/operator (on kind for example see docs)
use this for the redpanda_cluster.yaml
Main points for this repro/bug being
chartVersion: 5.9.2
+
connectors:
enabled: true
logging:
level: info
apiVersion: cluster.redpanda.com/v1alpha1
kind: Redpanda
metadata:
name: redpanda
spec:
chartRef:
chartVersion: 5.9.2
clusterSpec:
connectors:
enabled: true
logging:
level: info
external:
domain: customredpandadomain.local
auth:
sasl:
enabled: true
users:
- name: superuser
password: secretpassword
mechanism: SCRAM-SHA-512
storage:
persistentVolume:
enabled: true
storageClass: csi-driver-lvm-striped-xfs
- helm values
helm get values redpanda -n buggy --all
COMPUTED VALUES:
affinity: {}
auditLogging:
clientMaxBufferSize: 16777216
enabled: false
enabledEventTypes: null
excludedPrincipals: null
excludedTopics: null
listener: internal
partitions: 12
queueDrainIntervalMs: 500
queueMaxBufferSizePerShard: 1048576
replicationFactor: null
auth:
sasl:
enabled: true
mechanism: SCRAM-SHA-512
secretRef: redpanda-users
users:
- mechanism: SCRAM-SHA-512
name: superuser
password: secretpassword
clusterDomain: cluster.local
commonLabels: {}
config:
cluster:
default_topic_replications: 3
node:
crash_loop_limit: 5
pandaproxy_client: {}
rpk: {}
schema_registry_client: {}
tunable:
compacted_log_segment_size: 67108864
group_topic_partitions: 16
kafka_batch_max_bytes: 1048576
kafka_connection_rate_limit: 1000
log_segment_size: 134217728
log_segment_size_max: 268435456
log_segment_size_min: 16777216
max_compacted_log_segment_size: 536870912
topic_partitions_per_shard: 1000
connectors:
deployment:
create: false
enabled: false
test:
create: false
console:
affinity: {}
annotations: {}
automountServiceAccountToken: true
autoscaling:
enabled: false
maxReplicas: 100
minReplicas: 1
targetCPUUtilizationPercentage: 80
commonLabels: {}
config: {}
configmap:
create: false
console:
config: {}
deployment:
create: false
enabled: true
enterprise:
licenseSecretRef:
key: ""
name: ""
extraContainers: []
extraEnv: []
extraEnvFrom: []
extraVolumeMounts: []
extraVolumes: []
fullnameOverride: ""
global: {}
image:
pullPolicy: IfNotPresent
registry: docker.redpanda.com
repository: redpandadata/console
tag: ""
imagePullSecrets: []
ingress:
annotations: {}
className: null
enabled: false
hosts:
- host: chart-example.local
paths:
- path: /
pathType: ImplementationSpecific
tls: []
initContainers:
extraInitContainers: ""
livenessProbe:
failureThreshold: 3
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podLabels: {}
podSecurityContext:
fsGroup: 99
runAsUser: 99
priorityClassName: ""
readinessProbe:
failureThreshold: 3
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
replicaCount: 1
resources: {}
secret:
create: false
enterprise: {}
kafka: {}
login:
github: {}
google: {}
jwtSecret: ""
oidc: {}
okta: {}
redpanda:
adminApi: {}
secretMounts: []
securityContext:
runAsNonRoot: true
service:
annotations: {}
port: 8080
targetPort: null
type: ClusterIP
serviceAccount:
annotations: {}
automountServiceAccountToken: true
create: true
name: ""
strategy: {}
tests:
enabled: true
tolerations: []
topologySpreadConstraints: []
enterprise:
license: ""
licenseSecretRef: {}
external:
domain: customredpandadomain.local
enabled: true
service:
enabled: true
type: NodePort
fullnameOverride: ""
image:
pullPolicy: IfNotPresent
repository: docker.redpanda.com/redpandadata/redpanda
tag: ""
imagePullSecrets: []
license_key: ""
license_secret_ref: {}
listeners:
admin:
external:
default:
advertisedPorts:
- 31644
port: 9645
tls:
cert: external
port: 9644
tls:
cert: default
requireClientAuth: false
http:
authenticationMethod: null
enabled: true
external:
default:
advertisedPorts:
- 30082
authenticationMethod: null
port: 8083
tls:
cert: external
requireClientAuth: false
kafkaEndpoint: default
port: 8082
tls:
cert: default
requireClientAuth: false
kafka:
authenticationMethod: null
external:
default:
advertisedPorts:
- 31092
authenticationMethod: null
port: 9094
tls:
cert: external
port: 9093
tls:
cert: default
requireClientAuth: false
rpc:
port: 33145
tls:
cert: default
requireClientAuth: false
schemaRegistry:
authenticationMethod: null
enabled: true
external:
default:
advertisedPorts:
- 30081
authenticationMethod: null
port: 8084
tls:
cert: external
requireClientAuth: false
kafkaEndpoint: default
port: 8081
tls:
cert: default
requireClientAuth: false
logging:
logLevel: info
usageStats:
enabled: true
monitoring:
enabled: false
labels: {}
scrapeInterval: 30s
nameOverride: ""
nodeSelector: {}
post_install_job:
affinity: {}
enabled: true
podTemplate:
annotations: {}
labels: {}
spec:
containers:
- env: []
name: post-install
securityContext: {}
securityContext: {}
post_upgrade_job:
affinity: {}
enabled: true
podTemplate:
annotations: {}
labels: {}
spec:
containers:
- env: []
name: post-upgrade
securityContext: {}
securityContext: {}
rackAwareness:
enabled: false
nodeAnnotation: topology.kubernetes.io/zone
rbac:
annotations: {}
enabled: false
resources:
cpu:
cores: 1
memory:
container:
max: 2.5Gi
serviceAccount:
annotations: {}
create: false
name: ""
statefulset:
additionalRedpandaCmdFlags: []
additionalSelectorLabels: {}
annotations: {}
budget:
maxUnavailable: 1
extraVolumeMounts: ""
extraVolumes: ""
initContainerImage:
repository: busybox
tag: latest
initContainers:
configurator:
extraVolumeMounts: ""
resources: {}
extraInitContainers: ""
fsValidator:
enabled: false
expectedFS: xfs
extraVolumeMounts: ""
resources: {}
setDataDirOwnership:
enabled: false
extraVolumeMounts: ""
resources: {}
setTieredStorageCacheDirOwnership:
extraVolumeMounts: ""
resources: {}
tuning:
extraVolumeMounts: ""
resources: {}
livenessProbe:
failureThreshold: 3
initialDelaySeconds: 10
periodSeconds: 10
nodeSelector: {}
podAffinity: {}
podAntiAffinity:
custom: {}
topologyKey: kubernetes.io/hostname
type: hard
weight: 100
podTemplate:
annotations: {}
labels: {}
spec:
containers:
- env: []
name: redpanda
securityContext: {}
securityContext: {}
priorityClassName: ""
readinessProbe:
failureThreshold: 3
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
replicas: 3
securityContext:
fsGroup: 101
fsGroupChangePolicy: OnRootMismatch
runAsUser: 101
sideCars:
configWatcher:
enabled: true
extraVolumeMounts: ""
resources: {}
securityContext: {}
controllers:
createRBAC: true
enabled: false
healthProbeAddress: :8085
image:
repository: docker.redpanda.com/redpandadata/redpanda-operator
tag: v2.1.10-23.2.18
metricsAddress: :9082
resources: {}
run:
- all
securityContext: {}
startupProbe:
failureThreshold: 120
initialDelaySeconds: 1
periodSeconds: 10
terminationGracePeriodSeconds: 90
tolerations: []
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: ScheduleAnyway
updateStrategy:
type: RollingUpdate
storage:
hostPath: ""
persistentVolume:
annotations: {}
enabled: true
labels: {}
nameOverwrite: ""
size: 20Gi
storageClass: csi-driver-lvm-striped-xfs
tiered:
config:
cloud_storage_access_key: ""
cloud_storage_api_endpoint: ""
cloud_storage_azure_container: null
cloud_storage_azure_managed_identity_id: null
cloud_storage_azure_shared_key: null
cloud_storage_azure_storage_account: null
cloud_storage_bucket: ""
cloud_storage_cache_size: 5368709120
cloud_storage_credentials_source: config_file
cloud_storage_enable_remote_read: true
cloud_storage_enable_remote_write: true
cloud_storage_enabled: false
cloud_storage_region: ""
cloud_storage_secret_key: ""
credentialsSecretRef:
accessKey:
configurationKey: cloud_storage_access_key
secretKey:
configurationKey: cloud_storage_secret_key
hostPath: ""
mountType: emptyDir
persistentVolume:
annotations: {}
labels: {}
storageClass: ""
tests:
enabled: true
tls:
certs:
default:
caEnabled: true
external:
caEnabled: true
enabled: true
tolerations: []
tuning:
tune_aio_events: true
Anything else we need to know?
No response
Which are the affected charts?
No response
Chart Version(s)
helm -n buggy list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION redpanda buggy 2 2024-09-03 21:48:22.351235494 +0000 UTC deployed redpanda-5.9.2 v24.2.3 redpanda-controller buggy 1 2024-08-27 06:15:31.038271 +0100 BST deployed operator-0.4.28 v2.2.0-24.2.2
Cloud provider
..
JIRA Link: K8S-339
Setting tests.enabled=false should work around this issue for now. The problem stems from helm tests as indicated in the error message: "error":"template: redpanda/templates/tests/test-connector-via-console.yaml:150:24:
For a CRD:
apiVersion: cluster.redpanda.com/v1alpha1
kind: Redpanda
metadata:
name: redpanda
spec:
clusterSpec:
tests:
enabled: false
That being said, it's entirely possible that we discover other issues after disable the test step 😅
for me(support) disabling tests: got round the error.... and it created connectors
but for customer... it got round error ...but did not create the connectors..
and when using with "helm template" it wont generate the deployment...
Just tried the tests disable workaround, while it did fix the error, it seems to not deploy the connectors deployment at all.. Tried the same values file with "helm template" on version 5.9.1 and 5.9.2, 5.9.2 just won't generate the deployment...
It should be fixed in the latest Redpanda chart release v5.9.5
https://github.com/redpanda-data/helm-charts/releases/tag/redpanda-5.9.5
