flux2
flux2 copied to clipboard
fluxcd
Feature request: add option to limit watched namespaces by kustomization/helm controllers
Describe the bug
Hello!
Right now kustomization controller allows only two options: or to monitor current namespace, or to monitor all namespaces. I am building multi-tenant environment in single cluster, so it is crucial to me to limit the users for applying their objects into particular allow-list of namespaces. The naive approach is to make the several instances of flux gitops toolkit for each tenant restricted to allow-list of namespaces in both command line options and RBAC settings. I hope that I understood correctly the source code of kustomization, helm and source controllers and I refer to next options:
https://github.com/fluxcd/kustomize-controller/blob/1e84289623a4f4dc98b51a1cf9aaaac53ee64ccd/main.go#L80 https://github.com/fluxcd/helm-controller/blob/e2b988a11b3e35e3e702bc01b741181ac08f84af/main.go#L81 https://github.com/fluxcd/source-controller/blob/1475ecaa9bf22d5a1d2af9fc4cd9ab6fd36143f0/main.go#L103
Steps to reproduce
...
Expected behavior
...
Screenshots and recordings
No response
OS / Distro
N/A
Flux version
N/A
Flux check
N/A
Git provider
N/A
Container Registry provider
N/A
Additional context
No response
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Running multiple Flux instances per cluster is not advised. Please see here how to use Flux on multi-tenant clusters: https://github.com/fluxcd/flux2-multi-tenancy
@stefanprodan Thanks for your link. I surely will again read it and try to understand details.
I may add only my 5 cents that I have some fears that single flux installation per cluster is not very robust. I saw several cases when one incorrect kustomization (or one incorrect YAML in git repo) broke the whole synchronisation loop. This issue was /relatively/ difficult to investigate and to handle, so I had an idea that I may isolate each tenant from each other completely... Also on really big clusters (1000+ nodes with a hundreds Flux-related objects) sync is slow. I think I may fine-tune some settings, but again - the several flux instances will decrease the blast radius and as every flux will handle only several namespaces, the sync will be fast...
I saw several cases when one incorrect kustomization (or one incorrect YAML in git repo) broke the whole synchronisation loop.
I don't see how this is possible, each Kustomization is reconciled in isolation, if one fails it would not affect others unless you use dependsOn.
@stefanprodan clearly it is not advised, however, is it possible to utilise the scenario that @gecube is trying to achieve?
@gecube what approach did you choose? I have like the exact same scenario that I'll be working on, doing some research now
@nikodemjedynak-dnv Hi! I finished with the approach with the single flux instance per cluster. And I split the tenant, providing for each of them separate repository with the manifests.
@gecube how did you provide the ability to view the logs per tenant? @stefanprodan Is it possible to provide logs separation per tenant? I would like to achieve the multi-tenancy without using the CLI, is it possible?
This can now be archived using Flux sharding and --watch-label-selector. Docs here: https://fluxcd.io/flux/cheatsheets/sharding/