[BUG] rancher/rancher changes break UI tests

[nwmac]

Rancher Server Setup

• Rancher version: 2.9-head
• Installation option (Docker install/Helm Chart): Docker
• Proxy/Cert Details: none

Information about the Cluster

Tests are run against the local cluster.

User Information Tests are run as admin and standard user.

Describe the bug

Recent commits to rancher/rancher introduce numerous changes that break the UI automated test suite - see: https://github.com/rancher/dashboard/actions/runs/8782607195

Pinning the UI tests to use the container image tag v2.9-c9be13b09329bbee60a5f6419d500198f83c44d1-head allows the test to pass again.

A large number of tests are impacted, implying more than one breaking change here, example:

The following API calls fail with 500:

.v3/kontainerdrivers/rancher-windows-images
/v3/kontainerdrivers/rancher-image

The tests around extensions also fail.

To Reproduce Run the UI e2e test suite - or try and fetch the URLs above

Expected Result

UI tests should continue to pass.

Test automation should be added to prevent regressions going forward.

[richard-cox]

I've just brought up latest v2.9-head and it fails with lotd of these type of errors unable to access 'https://git.rancher.io/<various chart names>/': SSL certificate problem: unable to get local issuer certificate

We've also seen existing installs fail to fetch charts with

unable to access 'https://github.com/rancher/partner-extensions/': SSL certificate problem: unable to get local issuer certificate (see https://github.com/rancher/dashboard/issues/10868).

Could be related?

[mmartin24]

I've just brought up latest v2.9-head and it fails with lotd of these type of errors unable to access 'https://git.rancher.io/<various chart names>/': SSL certificate problem: unable to get local issuer certificate

We've also seen existing installs fail to fetch charts with

unable to access 'https://github.com/rancher/partner-extensions/': SSL certificate problem: unable to get local issuer certificate (see rancher/dashboard#10868).

Could be related?

FTR in case it helps, while testing this other item we noticed an fleet app upgrade from rancher 2.7 to 2.9 was faulty. It turned out Rancher needed flag "--set", "useBundledSystemChart=true" to work. Perhaps this issue may have been the reason: https://github.com/rancher/rancher/issues/44929

[IsaSih]

I've just brought up latest v2.9-head and it fails with lotd of these type of errors unable to access 'https://git.rancher.io/<various chart names>/': SSL certificate problem: unable to get local issuer certificate We've also seen existing installs fail to fetch charts with unable to access 'https://github.com/rancher/partner-extensions/': SSL certificate problem: unable to get local issuer certificate (see rancher/dashboard#10868). Could be related?

FTR in case it helps, while testing this other item we noticed an fleet app upgrade from rancher 2.7 to 2.9 was faulty. It turned out Rancher needed flag "--set", "useBundledSystemChart=true" to work. Perhaps this issue may have been the reason: #44929

It's important to highlight that the Rancher environment used on issue https://github.com/rancher/dashboard/issues/10868 was deployed with this flag "useBundledSystemChart=true" enabled. I'll update that in the issue body.

[gaktive]

Internal reference: SURE-8301

For tracking purposes, https://github.com/rancher/rancher/commit/325a0c5eb7dd65912c68abeea72e58bdf2bb097b was narrowed down as the PR that caused the issue.

PR https://github.com/rancher/rancher/pull/45246 fixes the issue.

For those following this ticket, @nwmac will check if E2E works afterwards and advise.

[gaktive]

QA did some testing already in https://github.com/rancher/rancher/issues/45199 -- not sure if that satisfies the issue here.

[Priyashetty17]

@MKlimuszka Why is this "To Test" for Frameworks QA? This seemed to have happened after rancher base image was transitioned to bci-micro because of which a few functionalities were broken in 2.9 - for instance, we couldn't add cloud credentials to Rancher either- https://github.com/rancher/rancher/issues/45199. The Hostbusters team fixed the issue few weeks ago. So, now the UI team might just need to retest with the latest 2.9-head version. cc @daviswill2 @brudnak

[gaktive]

@IsaSih has this issue been resolved since this was first found? Can we close this?

[daviswill2]

Per Priyashetty17's comment, https://github.com/rancher/rancher/issues/45205#issuecomment-2123010204, has the UI team retested with the latest 2.9-head version? @nwmac @IsaSih @mmartin24

[mmartin24]

Per Priyashetty17's comment, #45205 (comment), has the UI team retested with the latest 2.9-head version? @nwmac @IsaSih @mmartin24

I cannot check in our ci because ours uses another internal library that by default sets this flag on, but I'd say this is ok.

I have done clean deployment on 2.9-head (v2.9-bc816509bdb0ccab905459eadbcecaabcd51d521-head) via helm chart without explicitly setting --set useBundledSystemChart=true flag and looks good. I see no traces of error on cattle-fleet-system namespace either so it seems ok there as well.

Also performed upgrade 2.7 -head to 2.9-head mimicing the issue I noticed this with above and was ok as well.

Compare in screenshot before / after fix

I also did a run a few automated tests locally on latest 2.9-head and I saw no issues either.

[IsaSih]

Yes, it has been resolved. With https://github.com/rancher/dashboard/issues/10868, I retested this on top of the latest 2.9-head version. However, I found this issue https://github.com/rancher/rancher/issues/46019 that doesn't seem to be related.

https://github.com/rancher/rancher/assets/70187881/15190ff3-3b62-4cc0-8492-26c1bf5b0b56