[Monitoring v2] Links in Dashboard are un-clickable even the user has monitoring-ui-view permission

[jiaqiluo]

Setup

• Rancher version: 2.6.2
• Browser type & version: Chrome, Version 95.0.4638.54 (Official Build) (x86_64)

Describe the bug

User who is assigned the clusterRole monitoring-ui-view Should be able to click and open the URLs for Altermamgne, Grafana, Prometheus and etc. in the Monitoring Dashboard page in Cluster Explorer

To Reproduce

Steps:

• as admin, create a downstream cluster and enable monitoring v2
• as admin, create a local user user1 and assign it to be project-member of a project p1 in the cluster
• as admin, create the clusterRoleBinding (monitoring-ui-view, user-1 )
• log in as user-1, go to the cluster explorer UI -> monitoring tab

Result

• links on the monitoring dashboard are greyed out and not clickable
• but user-1 can open the grafana/Prometheus/etc URLs provided by the admin

Expected Result

These Links should be available

Screenshots

There is one failed call in the traffic:

Request URL: https://xxx.xxx.xx.xx/k8s/clusters/c-7nm22/v1/endpoints/cattle-monitoring-system/rancher-monitoring-alertmanager
Request Method: GET
Status Code: 403 Forbidden

Getting the endpoint for Alertmanager in the cattle-monitoring-system namespace is not needed for showing the links. Somehow the UI is trying to get some unnecessary resources which makes the links unavailable?

---

Update 1: New tests show that: the UI works as expected (links are available) when the user is assigned the view monitoring role via the old cluster management UI, but does not work if create the clusterRoleBinding (monitoring-ui-view, user-1 ) via kubectl. in both cases, UI sends a GET request to v1/endpoints/cattle-monitoring-system/rancher-monitoring-alertmanager and fail with 403 forbidden

[aiyengar2]

Identical to backend issue filed in https://github.com/rancher/rancher/issues/35311.

[gaktive]

Internal reference: SURE-4075. Updated https://github.com/rancher/rancher/issues/35311 too, where it feels like the bulk of the work would have to be done for monitoring.

[catherineluse]

What milestone should this be in?

[gaktive]

@catherineluse this is currently unscheduled though I'll sync up with @MKlimuszka since the related backend ticket shows more activity from the outside.

[ronhorton]

reopening checked in 2.8-head commit ID 436f4de

1. created downstream cluster
2. installed rancher-monitoring:103.0.0+up40.1.2
3. created standard user
4. added new user as project member with read-only access
5. set clusterrolebinding kubectl create clusterrolebinding view-monitoring-ui --clusterrole=monitoring-ui-view --user=u-xg7gf
6. logged in as user
7. did not observe error on login
8. naviagated to cluster > monitoring

403 error still shows on monitoring page

[mantis-toboggan-md]

Blocked by https://github.com/rancher/rancher/issues/43030

[gaktive]

Backend itself now listed as a blocker, https://github.com/rancher/steve/pull/132 to track for API updates.

[MbolotSuse]

@gaktive The PR that you linked is aiming at fixing rancher/rancher#43030 - I don't expect it alone to fix this issue.

[MbolotSuse]

Note on the linked rancher issue and associated steve fix (rancher/rancher#43030 and rancher/steve#132):

Steve was fixed to resolve the panic, but the endpoints won't be visible to users who are using the method outlined here to gain endpoint access (a clusterrolebinding to monitoring-ui-view). The reason why is this section of steve, which explicitly calls out the intended use case (i.e. get/list permissions on specific resource names at the cluster scope) as an invalid use case, that we don't want to return items for. This has existed for a long time (at least since July 2021), so I'm not eager to change it at this point in the release cycle, though I'm open to investigating it in the future.

Users can get around this by moving the cattle-monitoring-system namespace into a project (can be a special monitoring project) and giving users View Monitoring on that project (they still need read-only or higher access to another project to see the dashboard).

In short:

• The steve/rancher fix will fix the 500 error/panic, but still won't let these users see the monitoring links
• You can work around this issue with the following steps
  • After installing monitoring, move the cattle-monitoring-system namespace into a project
  • Give the users the monitoring-ui-view / View Monitoring role in the project containing cattle-monitoring-system
  • The user still needs to have read-only (or higher) in at least one project in that cluster

[gaktive]

Unblocked per https://github.com/rancher/rancher/issues/43030#issuecomment-1766597784 though I'm still wrapping my head around the workaround. Will add the release note label if this needs to go into the docs.

[gaktive]

Internal reference for docs: SURE-7044

and there's work to review the read-only permissions in the monitoring UI in SURE-7045 so there are limitations at present.

[mantis-toboggan-md]

Everything appears working as intended. A user with read-only project permission and the View Monitoring role shouldn't be able to use the links on the monitoring index (see https://github.com/rancher/rancher/issues/43030#issuecomment-1754056078). With the fix in https://github.com/rancher/steve/pull/132 visiting that page no longer causes a 500 error, the links are merely disabled as expected. I also confirmed that the workaround provided (moving the cattle-monitoring-system namespace) doesn't cause UI errors and the links become available.

[martyav]

This was labeled a release note but I'm having some difficulty parsing the comment thread. Can somebody provide a summary for the note?

[ronhorton]

A user with read-only project permission and the View Monitoring role shouldn't be able to use the links on the monitoring index (see https://github.com/rancher/rancher/issues/43030#issuecomment-1754056078).

[martyav]

Let me put it another way: If the UI is behaving as intended when users with such-and-such permissions can't view the links, why do we also have a work-around to circumvent that? Do we want to release note a work-around that gets around the intended behavior?

[martyav]

From Slack

@MbolotSuse

My recommendation would be: Release note that monitoring-ui-view permissions given through a clusteRroleBinding + read-only in a project is not sufficient for links to show up (known issue, I don't think we have an issue for this ATM) Add the workaround in https://github.com/rancher/dashboard/issues/4466#issuecomment-1766495804 to the release notes for the issue (move the monitoring-system namespace into a project, give monitoring-ui-view in that project)

[Wizmll]

Hey guys, Im having a similar issue (the links in Dashboard are un-clickable) except that Im the admin (Default admin) of the k3s cluster ? Does anyone have any idea why? Thanks