Traefik dashboard not accessible with port forwarding when setting ingressClass

[cstruct]

Welcome!

• [X] Yes, I've searched similar issues on GitHub and didn't find any.
• [X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you do?

Setting --providers.kubernetescrd.ingressclass=foo causes the dashboard to not be served on port 9000

Steps to reproduce:

1. Install Traefik helm install traefik traefik/traefik --set 'additionalArguments[0]=--providers.kubernetescrd.ingressclass=foo' --set 'logs.general.level=DEBUG'
2. Forward port 9000 kubectl port-forward (kubectl get pod -l app.kubernetes.io/instance=traefik -o name | head -n 1) 9000:9000
3. Try to access the dashboard curl 127.0.0.1:9000/dashboard/

Notice that the trailing slash is not missing.

If Traefik is installed without --set 'additionalArguments[0]=--providers.kubernetescrd.ingressclass=foo' the dashboard works as expected.

Adding a IngressRoute with the following service block:

name: "api@internal"
kind: "TraefikService"

successfully exposes the dashboard but this does not work for me since I want one internal Traefik to expose the dashboards of others.

What did you see instead?

Trying to access the Traefik dashboard gives 404 page not found.

What version of Traefik are you using?

2.4.3

What is your environment & configuration?

The only configuration set is --providers.kubernetescrd.ingressclass=foo and --log.level=DEBUG. Everything is deployed in a new empty Kubernetes cluster.

If applicable, please paste the log output in DEBUG level

time="2021-08-11T13:33:27Z" level=info msg="Configuration loaded from flags."
time="2021-08-11T13:33:27Z" level=info msg="Traefik version 2.4.13 built on 2021-07-30T15:06:29Z"
time="2021-08-11T13:33:27Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true,\"sendAnonymousUsage\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"metrics\":{\"address\":\":9100/tcp\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}},\"traefik\":{\"address\":\":9000/tcp\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}},\"web\":{\"address\":\":8000/tcp\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}},\"websecure\":{\"address\":\":8443/tcp\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"kubernetesIngress\":{},\"kubernetesCRD\":{\"ingressClass\":\"foo\"}},\"api\":{\"dashboard\":true},\"metrics\":{\"prometheus\":{\"buckets\":[0.1,0.3,1.2,5],\"addEntryPointsLabels\":true,\"addServicesLabels\":true,\"entryPoint\":\"metrics\"}},\"ping\":{\"entryPoint\":\"traefik\",\"terminatingStatusCode\":503},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"pilot\":{\"dashboard\":true}}"
time="2021-08-11T13:33:27Z" level=info msg="Stats collection is enabled."
time="2021-08-11T13:33:27Z" level=info msg="Many thanks for contributing to Traefik's improvement by allowing us to receive anonymous information from your configuration."
time="2021-08-11T13:33:27Z" level=info msg="Help us improve Traefik by leaving this feature on :)"
time="2021-08-11T13:33:27Z" level=info msg="More details on: https://doc.traefik.io/traefik/contributing/data-collection/"
time="2021-08-11T13:33:27Z" level=debug msg="Configured Prometheus metrics" metricsProviderName=prometheus
time="2021-08-11T13:33:27Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
time="2021-08-11T13:33:27Z" level=debug msg="Start TCP Server" entryPointName=traefik
time="2021-08-11T13:33:27Z" level=debug msg="Start TCP Server" entryPointName=metrics
time="2021-08-11T13:33:27Z" level=info msg="Starting provider *ingress.Provider {}"
time="2021-08-11T13:33:27Z" level=info msg="Starting provider *crd.Provider {\"ingressClass\":\"foo\"}"
time="2021-08-11T13:33:27Z" level=info msg="label selector is: \"\"" providerName=kubernetescrd
time="2021-08-11T13:33:27Z" level=info msg="Creating in-cluster Provider client" providerName=kubernetescrd
time="2021-08-11T13:33:27Z" level=info msg="Starting provider *acme.ChallengeTLSALPN {\"Timeout\":4000000000}"
time="2021-08-11T13:33:27Z" level=info msg="ingress label selector is: \"\"" providerName=kubernetes
time="2021-08-11T13:33:27Z" level=info msg="Creating in-cluster Provider client" providerName=kubernetes
time="2021-08-11T13:33:27Z" level=debug msg="Start TCP Server" entryPointName=web
time="2021-08-11T13:33:27Z" level=info msg="Starting provider *traefik.Provider {}"
time="2021-08-11T13:33:27Z" level=debug msg="Start TCP Server" entryPointName=websecure
time="2021-08-11T13:33:27Z" level=debug msg="Configuration received from provider internal: {\"http\":{\"routers\":{\"ping\":{\"entryPoints\":[\"traefik\"],\"service\":\"ping@internal\",\"rule\":\"PathPrefix(`/ping`)\",\"priority\":2147483647},\"prometheus\":{\"entryPoints\":[\"metrics\"],\"service\":\"prometheus@internal\",\"rule\":\"PathPrefix(`/metrics`)\",\"priority\":2147483647}},\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{},\"ping\":{},\"prometheus\":{}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"tls\":{}}" providerName=internal
time="2021-08-11T13:33:27Z" level=debug msg="Added outgoing tracing middleware ping@internal" routerName=ping@internal middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=traefik middlewareName=traefik-internal-recovery
time="2021-08-11T13:33:27Z" level=debug msg="Added outgoing tracing middleware prometheus@internal" middlewareType=TracingForwarder middlewareName=tracing routerName=prometheus@internal entryPointName=metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareType=Metrics entryPointName=web middlewareName=metrics-entrypoint
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=metrics middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareName=metrics-entrypoint middlewareType=Metrics entryPointName=traefik
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=metrics middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="No default certificate, generating one"
W0811 13:33:27.510609       1 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
W0811 13:33:27.512522       1 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
time="2021-08-11T13:33:27Z" level=debug msg="Configuration received from provider kubernetescrd: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=kubernetescrd
W0811 13:33:27.610485       1 warnings.go:70] networking.k8s.io/v1beta1 IngressClass is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 IngressClassList
W0811 13:33:27.611259       1 warnings.go:70] networking.k8s.io/v1beta1 IngressClass is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 IngressClassList
time="2021-08-11T13:33:27Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2021-08-11T13:33:27Z" level=debug msg="Configuration received from provider kubernetes: {\"http\":{},\"tcp\":{}}" providerName=kubernetes
time="2021-08-11T13:33:27Z" level=debug msg="No default certificate, generating one"
time="2021-08-11T13:33:27Z" level=debug msg="Added outgoing tracing middleware ping@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=ping@internal
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2021-08-11T13:33:27Z" level=debug msg="Added outgoing tracing middleware prometheus@internal" routerName=prometheus@internal middlewareType=TracingForwarder middlewareName=tracing entryPointName=metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareType=Metrics entryPointName=web middlewareName=metrics-entrypoint
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareName=metrics-entrypoint middlewareType=Metrics entryPointName=metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareType=Metrics entryPointName=metrics middlewareName=metrics-entrypoint
time="2021-08-11T13:33:27Z" level=debug msg="Creating middleware" middlewareName=metrics-entrypoint entryPointName=traefik middlewareType=Metrics
time="2021-08-11T13:33:27Z" level=debug msg="No default certificate, generating one"
time="2021-08-11T13:33:28Z" level=debug msg="No default certificate, generating one"
time="2021-08-11T13:33:28Z" level=debug msg="Added outgoing tracing middleware prometheus@internal" middlewareType=TracingForwarder entryPointName=metrics routerName=prometheus@internal middlewareName=tracing
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" entryPointName=metrics middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2021-08-11T13:33:28Z" level=debug msg="Added outgoing tracing middleware ping@internal" entryPointName=traefik routerName=ping@internal middlewareName=tracing middlewareType=TracingForwarder
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" middlewareName=metrics-entrypoint middlewareType=Metrics entryPointName=metrics
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" middlewareName=metrics-entrypoint middlewareType=Metrics entryPointName=traefik
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" middlewareType=Metrics middlewareName=metrics-entrypoint entryPointName=web
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" middlewareType=Metrics entryPointName=websecure middlewareName=metrics-entrypoint
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" entryPointName=metrics middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:28Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=metrics-entrypoint middlewareType=Metrics
time="2021-08-11T13:33:28Z" level=debug msg="No default certificate, generating one"
time="2021-08-11T13:33:28Z" level=debug msg="No default certificate, generating one"
time="2021-08-11T13:33:46Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2021-08-11T13:33:46Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetes

[cstruct]

Why was this issue moved? This is not specific to helm. I only used helm in the issue to make it easily reproducible.

[texano00]

Have you configured this?

  ingressRoute:
    dashboard:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: '{your-class}'

[cstruct]

Thank you @texano00, and sorry for the late response. Adding the ingress class annotation was the missing piece. (I feel like I should have seen that was the issue myself when setting everything up without Helm but alas I did not :sweat_smile:)

To get my full setup working each Traefik v2 ingress controller needed to specify itself in that annotation. Then a ClusterIP service exposing 9000 for each controller could be added that our internal ingress controller could add IngressRoutes for.

With this information I wouldn't call this a bug but it's up for discussion if documentation is lacking or if the chart should add this annotation automatically when setting ingress class. Is that something that we might want to do or should I just close this issue?