harbor Cannot use existingSecret for external redis

When I configure an externalSecret from external redis I have the following errors:

upgrade.go:144: [debug] preparing upgrade for harbor
Error: UPGRADE FAILED: YAML parse error on harbor/templates/jobservice/jobservice-cm.yaml: error converting YAML to JSON: yaml: line 20: found character that cannot start any token
helm.go:84: [debug] error converting YAML to JSON: yaml: line 20: found character that cannot start any token
YAML parse error on harbor/templates/jobservice/jobservice-cm.yaml
helm.sh/helm/v3/pkg/releaseutil.(*manifestFile).sort
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/releaseutil/manifest_sorter.go:146
helm.sh/helm/v3/pkg/releaseutil.SortManifests
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/releaseutil/manifest_sorter.go:106
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/action/action.go:170
helm.sh/helm/v3/pkg/action.(*Upgrade).prepareUpgrade
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/action/upgrade.go:236
helm.sh/helm/v3/pkg/action.(*Upgrade).RunWithContext
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/action/upgrade.go:145
main.newUpgradeCmd.func2
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/upgrade.go:201
github.com/spf13/cobra.(*Command).execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:940
github.com/spf13/cobra.(*Command).ExecuteC
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:1068
github.com/spf13/cobra.(*Command).Execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:992
main.main
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
	/usr/lib/golang/src/runtime/proc.go:267
runtime.goexit
	/usr/lib/golang/src/runtime/asm_amd64.s:1650
UPGRADE FAILED
main.newUpgradeCmd.func2
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/upgrade.go:203
github.com/spf13/cobra.(*Command).execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:940
github.com/spf13/cobra.(*Command).ExecuteC
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:1068
github.com/spf13/cobra.(*Command).Execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:992
main.main
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
	/usr/lib/golang/src/runtime/proc.go:267
runtime.goexit
	/usr/lib/golang/src/runtime/asm_amd64.s:1650

From what I can tell this is actually an issue with b64dec from helm.

In _heplers.tpl on line 161 it grabs the existing secret data and decodes it to use when making the config map.

{{- define "harbor.redis.pwdfromsecret" -}}
  {{- (lookup "v1" "Secret"  .Release.Namespace (.Values.redis.external.existingSecret)).data.REDIS_PASSWORD | b64dec -}}
{{- end -}}

At one point in my testing I was also able to generate the manifest and everything looked fine, the password was in the url, but it still threw the error. After I replace the lookup with print "base64string" | b64dec it still caused the error. I also dont believe that it was an issue with my password since it works just fine if I do print "password".

Mar 25 '24 01:03 CrimsonFez

Duplicate of goharbor/harbor-helm#1641 See my comment there for exact explanation

Mar 27 '24 18:03 Kajot-dev

This is not a duplicate because I'm not using helm template, I'm using helm upgrade/install.

Mar 27 '24 20:03 CrimsonFez

Sorry, you're right. Can you render the invalid YAML and paste it here? (you should be able with --debug option)

Mar 27 '24 20:03 Kajot-dev

It doesn't output any yaml. Just the error

❯ helm -n harbor upgrade --install harbor harbor/harbor --version 1.14.1 --values values.yaml --debug
history.go:56: [debug] getting history for release harbor
upgrade.go:144: [debug] preparing upgrade for harbor
Error: UPGRADE FAILED: YAML parse error on harbor/templates/jobservice/jobservice-cm.yaml: error converting YAML to JSON: yaml: line 20: found character that cannot start any token
helm.go:84: [debug] error converting YAML to JSON: yaml: line 20: found character that cannot start any token
YAML parse error on harbor/templates/jobservice/jobservice-cm.yaml
helm.sh/helm/v3/pkg/releaseutil.(*manifestFile).sort
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/releaseutil/manifest_sorter.go:146
helm.sh/helm/v3/pkg/releaseutil.SortManifests
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/releaseutil/manifest_sorter.go:106
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/action/action.go:170
helm.sh/helm/v3/pkg/action.(*Upgrade).prepareUpgrade
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/action/upgrade.go:236
helm.sh/helm/v3/pkg/action.(*Upgrade).RunWithContext
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/pkg/action/upgrade.go:145
main.newUpgradeCmd.func2
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/upgrade.go:201
github.com/spf13/cobra.(*Command).execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:940
github.com/spf13/cobra.(*Command).ExecuteC
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:1068
github.com/spf13/cobra.(*Command).Execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:992
main.main
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
	/usr/lib/golang/src/runtime/proc.go:267
runtime.goexit
	/usr/lib/golang/src/runtime/asm_amd64.s:1650
UPGRADE FAILED
main.newUpgradeCmd.func2
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/upgrade.go:203
github.com/spf13/cobra.(*Command).execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:940
github.com/spf13/cobra.(*Command).ExecuteC
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:1068
github.com/spf13/cobra.(*Command).Execute
	/usr/share/gocode/src/github.com/spf13/cobra/command.go:992
main.main
	/builddir/build/BUILD/helm-3.11.1/_build/src/helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
	/usr/lib/golang/src/runtime/proc.go:267
runtime.goexit
	/usr/lib/golang/src/runtime/asm_amd64.s:1650

Mar 31 '24 18:03 CrimsonFez

@CrimsonFez could you please share with us which version of harbor-helm you are using when facing this issue? Thanks

Apr 18 '24 02:04 zyyw

1.14.1

Apr 18 '24 02:04 CrimsonFez

Hi @CrimsonFez ,

Could you try to upgrade/helm using redis password instead of existingSecret to narrowdown the error scope
Please share your values.yaml and check if you quote the fields.

Apr 19 '24 06:04 MinerYang

I currently deploy with the password in my values, so that works. This issue is present on 1.14.0, 1.14.1, and 1.14.2.

Here are my values:

expose:
  tls:
    certSource: secret
    secret:
      secretName: harbor-ingress
  ingress:
    hosts:
      core: harbor.example.com
    harbor:
      annotations:
        cert-manager.io/cluster-issuer: letsencrypt-prod-http

externalURL: https://harbor.example.com

core:
  replicas: 2

registry:
  replicas: 2

portal:
  replicas: 2

persistence:
  resourcePolicy: "keep"
  persistentVolumeClaim:
    trivy:
      storageClass: "rbd-ssd-r3"
    registry:
      storageClass: "cephfs-fast"
      accessMode: "ReadWriteMany"
      size: "50Gi"
  imageChartStorage:
    type: filesystem
    disableredirect: true
    filesystem:
      rootdirectory: /storage
      maxthreads: 100

database:
  type: external
  external:
    host: harbor-pg-primary
    username: harbor
    coreDatabase: harbor
    existingSecret: harbor-pg-pguser-harbor
    sslmode: "require"

redis:
  type: external
  external:
    addr: keydb:6379
    existingSecret: redis-password

jobservice:
  jobLoggers:
    - database

logLevel: error

Apr 19 '24 15:04 CrimsonFez

I have the same issue, install redis with the bitnami chart, install with the custom values:

auth:
  enabled: true
  sentinel: true
  existingSecret: "redis-secret"
  existingSecretPasswordKey: "REDIS_PASSWORD"

copy secret to harbor namespace, install harbor chart with:

redis:
  type: external
  external:
    addr: "redis-node-0.redis-headless.redis.svc.cluster.local:26379......"
    sentinelMasterSet: "mymaster"
    existingSecret: "redis-secret"

Errors:

**sentinel.go:514: sentinel: GetMasterAddrByName master="mymaster" failed: NOAUTH Authentication required.
failed to ping redis+sentinel://:[email protected]:26379**

Change the redis chart to:

auth:
  enabled: true
  sentinel: false

make no other changes, and Harbor starts working straight away: sentinel.go:661: sentinel: new master="mymaster" addr="redis-node-0.redis-headless.redis.svc.cluster.local:6379"

I would have expected it to fail since I'm still telling Harbor to provide a password in the Helm chart with existingSecret. If I run: kubectl exec -it redis-cli-pod -- redis-cli -h redis-node-0.redis-headless.redis.svc.cluster.local -p 26379 -a $REDIS_PASSWORD SENTINEL get-master-addr-by-name mymaster

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
AUTH failed: ERR AUTH <password> called without any password configured for the default user. Are you sure your configuration is correct?
1) "redis-node-3.redis-headless.redis.svc.cluster.local"
2) "6379"

May 14 '24 03:05 Vertiwell

Ah, yes Harbor does not support auth for sentinel, only for redis. This is a long standing issue, see https://github.com/goharbor/harbor/issues/14757 (although is should be relatively easy to fix)

Jun 21 '24 10:06 Kajot-dev

This issue was not originally related to sentinel authentication. This is still an issue with helm. Do you mind moving it back to harbor-helm? Thanks

Jun 21 '24 14:06 CrimsonFez

@CrimsonFez Sorry, I was mistaken by the latest comment

Jun 21 '24 15:06 Kajot-dev

Root cause: Harbor components (core, jobservice, trivy, exporter) require Redis password embedded in the connection URL at template time. This breaks both helm template (GitOps) and proper secretKeyRef usage.

Registry works because it reads password from env var (REGISTRY_REDIS_PASSWORD) separately from the URL.

Fix needed: Add env var support for Redis password in core, jobservice, trivy, exporter - same pattern registry uses.

Opening separate feature request for this.

Ref: goharbor/harbor-helm#2291, goharbor/harbor-helm#1641

Dec 23 '25 01:12 bupd