Project owner fails to enable project monitoring in v2

[gaktive]

Internal reference: SURE-4734 Reported in 2.6.5

Issue description: If you select a project in the dropdown list, you can set up your installation. But if you click on the selected project a second time, the UI doesn't show up anything anymore.

Repro steps:

• Enable the project monitoring by installing the corresponding charts
• Navigate to Cluster-Monitoring-Project Monitors
• Click on create (https://$IP_ADDRESS/dashboard/c/c-2q8b6/monitoring/helm.cattle.io.projecthelmchart/create)
• Select the project, it will display AlertManager, Grafana,federation tabs
• Again select the same Project

Workaround: Is workaround available and implemented? yes What is the workaround: select other project again click on same project.

Actual behavior: it is not showing AlertManager, Grafana,federation tabs if we select the same project again. Able to reproduce the issue in my environment. Not sure is it intentionally implemented in this way or its an UI bug.

Expected behavior: User expects it should show same UI components if we select same project again.

[gaktive]

Possibly tied to SURE-4692 as well.

Issue description: with the Prometheus federator installed, the project owner cannot install project monitoring…they get this error "Method POST Not Supported"

Repro steps:

• Install the Prometheus federator
• try to enable the project monitor with account having project owner role.

Workaround: Is workaround available and implemented? yes What is the workaround: able to enable with admin account.

Actual behavior: It is showing "Method POST Not Supported" when we try to enable the Project monitoring with project owner role.

Expected behavior:

it should enable project monitoring for an account having project owner roles, however if the account(user) not have proper permission, then it should show a proper error messages saying "permission not enough to enable the project monitoring".

Additional notes: Via someone's repro in 2.6.5, it seems the issue is coming before installing the ProjectHelmChart/ project monitor operator. After installing ProjectHelmChart, they're not not able to reproduce; it is pushing the CRD,RBAC role binding properly.

[gaktive]

Additional update from SURE-4692:

Users still cannot enable project monitoring....when they try to install the helm_project_operator they receive the following: Error: INSTALLATION FAILED: rendered manifests contain a resource that already exists. Unable to continue with install: could not get information about the resource PodSecurityPolicy "helm-project-operator-psp" in namespace "": podsecuritypolicies.policy "helm-project-operator-psp" is forbidden: User "u-q6str" cannot get resource "podsecuritypolicies" in API group "policy" at the cluster scope

[ghost]

I don't believe 4692 is related (that one seems more RBAC related than anything) but I don't seem to be able to reproduce this issue as described. My steps are as follows:

1. Create new cluster for Project Monitoring Test (RKE2) using Rancher 2.6.6
2. Install Monitoring Chart
3. Install Prometheus Federator (uncheck "Enable Embedded Helm Controller" as this is an RKE2 cluster)
4. Create "projectA" project and "namespace-a" namespace, notice "cattle-project-" namespace is also created
5. Create "projectB" project and "namespace-b" namespace, notice "cattle-project-" namespace is also created
6. Navigate to "Monitoring > Project Monitor"
7. Click "Create" button in top-right corner
8. Select "projectA" in "Project" dropdown
9. Reopen "Project" dropdown and select "projectA" again
10. Noticed that form tabs ("Alertmanager", "Grafana", "Federation") are still present.

Also tried: Reproducing when "projectA" already has Project Monitoring installed Reproducing using "projectB" when "projectA" already exists Reproducing with "default" project

[ghost]

Jumped on a call with Suresh to reproduce. We were unable to reproduce in 2.6.7 but we were able to reproduce in 2.6.5. It looks like the fix for this was a side-effect of another fix in 2.6.7 and should be resolved by upgrading to 2.6.7 on release. Putting into "Test" to independently verify that the issue is resolved in the upcoming release.

[AndrewHoffmanQA]

AndrewHoffmanQA said: retest: Passed

retest cluster information rancher- 2.6.7-rc5 rke2 version- v1.24.2+rke2r1

I followed @Sean-McQ repro steps as well as some adhock testing, I can confirm that this bug is no longer reproducible. closing ticket.

[nwmac]

This is still being reported as an issue

[gaktive]

SURE-4692 has become active again, so putting this back on our radar for triage.

[gaktive]

@catherineluse once you've tackled over your other priority tickets, see what you can do here based on past efforts.

[gaktive]

Update from @victorcasado based on one impacted user based on questions from @Sean-McQ:

• Which version of monitoring and project-federator are installed?

Monitoring is v100.1.3+up19.0.3 and Project Federator is prometheus-federator:0.1.1

• Verify rancher version in the dashboard (as mentioned, it’s 2.6.8 but just verifying amidst all the prior screenshots from 2.6.5)

2.6.8 confirmed

• Upon the problem occurring, bring up the console and check for console errors

where to look exactly? which pod?

• As well, check the network tab when reproducing and look for red rows, capturing any errors.

based on HAR file (which needs scrubbing), only one red line: 403 spotted when trying to access /k8s/clusters/c-8kc8w/v1/helm.cattle.io.projecthelmcharts

• Does this work for users with different permissions? It’s mentioned in a comment that a project member can’t but can project owners and admins enable without issue?

project owners CAN enable monitoring (this is NEW)

• Can you get me a list of namespaces in the projects that reproduce these errors? Is this all namespaces or some?

all

[ghost]

The console in question here is in the web-browser's developer tools. In chrome, you can follow these steps to open the console and get the information. https://www.webucator.com/article/how-to-open-google-chromes-javascript-console/

The first thing that jumps out at me is the 403 error you're seeing on /k8s/clusters/c-8kc8w/v1/helm.cattle.io.projecthelmcharts. A 403 (which refers to a "Forbidden" response) means that the user is authenticated properly but lacks the permissions to access the resource in question. Basically, this indicates to me that this is a permissions issue, if we can get somebody from the backend to clarify which permissions might be missing to cause a 403 on this resource, we can check on the frontend and then disable the action if the permission isn't present.

[gaktive]

Update from @MKlimuszka:

https://github.com/rancher/prometheus-federator/blob/main/charts/prometheus-federator/0.1.1/charts/helmProjectOperator/templates/clusterrole.yaml These are the permissions that we use by default. If the UI has more questions about this, let [the backend at Team 3] know. If the reporter wants a member to be able to have more permissions, they need to assign a role binding attached to one of those cluster roles within the project registration namespace. Also, based on one of the recent comments on this ticket, project owners can enable project monitoring, so it sounds like the original issue reported in this ticket is fixed and if there is a new bug, a new ticket should be opened.

I need to reread this and see how https://github.com/rancher/dashboard/pull/7294 maps to addressing the situation.

[AndrewHoffmanQA]

retest passed. moving to done column.

tested on 2.7head 38979f8

retest video attached

https://user-images.githubusercontent.com/89030218/205372828-f61c6cd0-8be8-48a8-94ab-dffa8945c1b4.mp4