Notification dispatch failure errors

[kalyanteja19]

Hi Team,

We are facing issue with notifications dispatch. We are trying to send flux notifications to alert manager that is enabled with basic authentication.

{"level":"debug","ts":"2025-06-24T11:51:36.214Z","logger":"events","msg":"failed to dispatch notification for Kustomization/flux-system/k8s-common-modules: failed to initialize notifier for provider 'alertmanager': failed to initialize notifier: invalid Alertmanager URL https://xxxxxx:xxxxx:alertmanager.cloud.com/api/v2/alerts/: 'parse "https://xxx:xxxx:alertmanager.cloud.com/api/v2/alerts/": net/url: invalid userinfo'","type":"Warning","object":{"kind":"Alert","namespace":"flux-system","name":"alertmanager-alerts","uid":"b0adf599-b7d9-4089-aabe563d156684d9","apiVersion":"notification.toolkit.fluxcd.io/v1beta3","resourceVersion":"37063594"},"reason":"NotificationDispatchFailed"}

Please be informed that password contains special characters like @,# and tried multiple ways to escape special characters, but its not working. Can you suggest on this issue?

Version flux 2.5.1 and Kubernetes version is 1.31.8

[matheuscscp]

This is how your secret for alertmanager with basic auth should look like (docs):

apiVersion: v1
kind: Secret
metadata:
  name: alertmanager-address
  namespace: default
stringData:
    address: https://<username>:<password>@<alertmanager-hostport>/api/v2/alerts/

Your URL is separating the basic auth part from the host with :, not a @:

https://xxx:xxxx:alertmanager.cloud.com/api/v2/alerts/

[kalyanteja19]

This is how your secret for alertmanager with basic auth should look like (docs):

apiVersion: v1 kind: Secret metadata: name: alertmanager-address namespace: default stringData: address: https://:@/api/v2/alerts/ Your URL is separating the basic auth part from the host with :, not a @:

https://xxx:xxxx:alertmanager.cloud.com/api/v2/alerts/

@matheuscscp My bad, I did not mention format properly, we are using @ separator only. Can you suggest?

[matheuscscp]

Yeah I also tried a few things here... I'm not sure how to escape either. Your user/pass just have to comply with Go's url.ParseRequestURI():

	_, err := url.ParseRequestURI(hookURL)
	if err != nil {
		return nil, fmt.Errorf("invalid Alertmanager URL %s: '%w'", hookURL, err)
	}

You can try a few different things running the Go snippet above. When you find a suitable password, reconfigure your basic auth settings.

[matheuscscp]

For the current version of Flux, your only solution is to comply with Go's url.ParseRequestURI(), or change your authentication to Bearer Token: https://fluxcd.io/flux/components/notification/providers/#prometheus-alertmanager

For the next Flux version, we could easily fix this by using the username and password fields from .spec.secretRef besides address and properly .SetBasicAuth() in the request, which sets the basic auth creds directly in the HTTP header, without the URL parsing problem. Would you like to contribute this feature?

[kalyanteja19]

Hi @matheuscscp , Great to hear! We will wait for the next upgrade.

[matheuscscp]

We will wait for the next upgrade.

?

[kalyanteja19]

We will wait for the next upgrade.

?

I am afraid to say that may not be able to contribute but I can do feature testing.

[kalyanteja19]

Hi @matheuscscp , Good day! Can you please suggest if this issue can be fixed soon?

[matheuscscp]

@kalyanteja19 This will land in Flux 2.7, end of September.