Unable to use traefik 3.4.3 IngressRoute to direct to kong proxy port 80 or 443

[majones-services]

What happened?

I've been working for several days to finish a project related to k3s. The dashboad can be installed without issue and port forwarding will work without issue. However, with the range of users expected, it's not realistic to expect everyone to remember to 1) have kubectl installed and 2) all the details of the specific commands.

What did you expect to happen?

I expected that I would be able to craft a ingressroute.yaml that would perform as expected vs. what actually did

How can we reproduce it (as minimally and precisely as possible)?

1. curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.31.5+k3s1" K3S_CONFIG_FILE=/etc/rancher/k3s/config.yaml sh -s - server --cluster-init --disable=traefik
2. The k3s installation is "hardened" per the hardening guide (less the dns recommendations). I'm happy to share the exact yaml needed. In the end, the updates apply a pod security admission plan that is restrictive: normal things like can't run as root and namespace -> namespace restrictions
3. After install of k3s, using helm, install traefik: helm install traefik traefik/traefik -f values.yaml --namespace kube-system. The yaml file sets up the certificate resolver and the needed secrets
4. I install the dashboard via helm: helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kube-dashboard

Here is the ingressroute yaml that I use:

apiVersion: traefik.io/v1alpha1 kind: IngressRoute metadata: name: kubernetes-dashboard-ingress namespace: kube-dashboard spec: entryPoints:
- websecure routes:
- kind: Rule match: Host(kubernetes-dashboard.XXXXXXX.org) && PathPrefix(/) priority: 10
services:
- name: kubernetes-dashboard-kong-proxy port: 443 tls:
certResolver: cloudflare

This ingress clearly terminates tls at the edge and forwards the traffic un-encrypted to kong. I also install the dashboard with http enabled:

proxy: type: ClusterIP http: enabled: true

Anything else we need to know?

Again, port-forward works perfectly but with traefik user experiences the following:

1. Login screen pops up, no issue with the Bearer Token field ready .

2. user enters the bearer token and then the user is presented with a 404 error

I can see in the traefic log and in kong log that the client browser asked for and received resources like /settings /config and other resource needed. However, /api/v1/me failed with 404 and every other /api call the same. No error or message in any log. traefik or kong.

I've tried installing the dashboard with http enabled to no avali. Also work thru disabling Kong and only using traefik to route, following another issues process but got the same 404 errors no matter what I've done.

What browsers are you seeing the problem on?

Chrome

Kubernetes Dashboard version

7.11.1

Kubernetes version

v1.31.5+k3s1

Dev environment

No response

[Blu53E]

Same - following issue.

Been trying to solve for this for the past few days. There's no documentation of this type of access on the project page. Tried several solutions found on the web but they just don't work. I keep getting a 404 as well and have yet to solve for it. So, again - not sure if this is no longer supported or if it's just simply due to under documentation.

I too am using k3s. kubectl proxy / port-forward works, nodeport works but unable get an ingress with custom url to work.

[floreks]

I'll try to find some time to describe simple local k3s setup with traefik. I suspect that it somehow drops some headers from the request. Should be a simple solve.

[helong001]

Same issue

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: k8s-dashboard-ingressroute
  namespace: kubernetes-dashboard
spec:
  entryPoints:
    - websecure  
  routes:
    - match: Host(`dashboard.k8s.local`)  
      kind: Rule
      services:
        - name: kubernetes-dashboard-kong-proxy  
          port: 443  
          scheme: https  

When deploying Kubernetes Dashboard using the Helm Chart, the dashboard communicates internally through the Kong Proxy, which primarily handles authorization APIs and metric forwarding for the dashboard’s internal components.

However, the following issues occur during Traefik’s communication with Kong Proxy: • When Traefik accesses Kong Proxy via the websecure (HTTPS) entry point, if Traefik is not properly configured with TLS certificates, requests fail with an internal server error. • When accessing Kong Proxy via the web (HTTP port 80) entry point, the Kong Proxy must have HTTP support enabled as follows:

kong:
  proxy:
    http:
      enabled: true

Even with HTTP enabled, when using this insecure route, although the login token is correct, the login process returns Unauthorized (401): Invalid credentials provide.

[gaileys]

I'm seeing exactly the same issue.

[wuast94]

i fixxed it with:

apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
  name: dashboard-transport
  namespace: kubernetes-dashboard
spec:
  insecureSkipVerify: true
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`sub.domain.tld`)
      kind: Rule
      middlewares:
        - name: chain-auth
          namespace: traefik
      services:
        - name: kubernetes-dashboard-kong-proxy
          port: 443
          scheme: https
          serversTransport: dashboard-transport

maybee it helps someone

[Antebios]

This worked for me!! Everything in the dashboard is working.

---
apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
  name: k8sdashboard-insecure-transport
  namespace: kubernetes-dashboard
spec:
  insecureSkipVerify: true

---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: k8sdashboard-dev-private
  namespace: kubernetes-dashboard
  annotations:
    kubernetes.io/ingress.class: traefik-internal
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`k8sdashboard-dev.macross.com`) && PathPrefix(`/`)
      services:
        - kind: Service
          name: k8sdash-kong-proxy
          namespace: kubernetes-dashboard
          port: 443
          serversTransport: k8sdashboard-insecure-transport
    - kind: Rule
      match: Host(`k8sdashboard-dev.macross.com`) && PathPrefix(`/api`)
      services:
        - kind: Service
          name: k8sdash-kong-proxy
          namespace: kubernetes-dashboard
          port: 443
          serversTransport: k8sdashboard-insecure-transport
  tls:
    secretName: k8sdashboard-private-tls