Support for multi-tenant clusters

[stephen-harris]

Apologies if this is already supported, but it doesn't appear to be be (and I couldn't see anything in the documentation)

Is your feature request related to a problem? Please describe. We run a multi-tenant cluster where different teams are namespaced and each team has only permissions only to their namespace(s). We'd like for teams to be able to create and a manage test resources themselves in their own namespaces (without the need to run an instance of testkube in each namespace). Additionally, we'd like to be able to configure slack-notifications at a team (or even test level).

Describe the solution you'd like

• Teams can create test resources in any namespace (that they have permissions for)
• Teams can execute tests in namespaces they own
• Test executions occur in the team's namespace
• Teams can view tests & test executions in their own namespace (at least with a namespace filter to help find tests)
• Teams cannot execute tests in namespaces they don't have access to
• Teams can configure a slack channel at a test level

Describe alternatives you've considered Nothing so far

Additional context Having support for multiple teams in the UI would also be great. e.g. supporting OIDC authentication to identify a user in a OIDC group and allowing configuration to map that group to a namespace / permissions.

[vsukhin]

Hey @stephen-harris

1. Slack notifications are under the testing right now (they are event based, but, probably needs to be adjusted for a team level @nicufk)
2. We don't support multi-tenancy right now, you need to install Testkube in each namespace, but we have already had a request to support vcluster, so this will be definitely a required feature for multi team company
3. We have supported OAuth for UI for Github provider right now and planning to support it for CLi as well. As obvious this feature has a lot of things to be improved )

[exu]

Like this feature very much - it'll for sure help bigger players where there is a lot of teams working on single product. We should check what's needed/missing in testkube to be able to implement such functionality.

@TheBrunoLopes FYI ^^^

[mkoertgen]

I would love testkube to discover tests/suites from other namespaces as well.

One use case for us is that we like to use namespaces for isolating deployment environments (such as 'dev', 'stage', 'production', ...).

Starting tests against services deployed to these namespaces allows for easy reuse of the test-manifests (no service-fqdn needed, no passing variables needed).

Also we would prefer managing tests/suites with kubernetes manifests instead of the testkube-cli / kubectl/krew-plugin. This way we can use GitOps-Tools like FluxCD with ease.

For notifications, the slack notification option is fine and sufficient for us, no need for a special fluxcd-notification-provider. We would either use the provided Slack-feature and/or Prometheus/Alertmanager for notifying on failed tests. The latter would make navigating to the respective tests in the testkube-dashboard harder, yet.

[chinthanep-asml]

Multi-tenancy support would be really good. Is there any recent update on this feature?

[vsukhin]

We're working on this feature. @TheBrunoLopes can comment it

[chinthanep-asml]

@vsukhin @TheBrunoLopes Thanks. I would be very happy to get info about it.

[TheBrunoLopes]

Hello @chinthanep-asml the way we are tackling "multi tenancy" is that we are working on making Testkube namespace scoped, in a way that allows you to have different teams using different Testkube instances in different namespaces. Is there any other multi tenancy feature you would like to have ? I would love to know more about your use case. Please feel free to go on a quick call with me using this link -> https://calendly.com/bruno-at-kubeshop/30min-1

[chinthanep-asml]

@TheBrunoLopes : Thanks for your reply. I have booked an appointment to discuss this with you on Monday.

[gberche-orange]

In my team, we are using one namespace per component installed, resulting in approx 50 namespaces in each K8S cluster.

We're generally adopting K8S controllers we can use across all or our namespaces (e.g. kyverno, flux, cert-manager, carvel secret-gen etc...)

The current requirement of installing Testkube oss (680 MB ram) in each namespace that declares and runs tests is too expensive for our team. This is preventing growth in Testkube usage in our team and widespread adoption in our company for other teams.

[vsukhin]

hey @gberche-orange No worries, the idea iof Testkube is to fit different user needs. Current approach will remain as default one with cluster wide access to all namespaces, but for those, who need to restrict access for each company team, we will provide options for multi namespace support. @TheBrunoLopes @ypoplavs @exu