azure-service-operator
azure-service-operator copied to clipboard
Azure
Install Selected CRDs only by default
Instead of blithely installing all custom resource definitions (CRDs) when the operator is deployed, we should have an approach that allows users to install only the CRDs they need.
This will help to avoid any performance issues that might be caused by installing hundreds of CRDs by only installing the dozen or so needed for a specific context.
Suggested approach is some kind of configuration file that gives control. Some CRDs will be enabled by default, but most will be disabled.
We likely would want some dependency information included - e.g. if you want to use SQL Server, you probably want Networking and AAD as well.
We still want to do this. We need to do some design about what exactly it'll look like though
installation is taking me 36min to complete. This is way to much time.
if we can have the basic chart and specify the crds to enable that will be great!
36m is way too long. Can you share more details about the environment you're installing into? k8s version, any Validating or Mutating webhooks configured which could be acting on the CRDs and slowing things down?
Also are you installing with the Helm chart or the raw YAML?
It takes maybe 1m for me to install the whole Helm chart on a fresh cluster. So my guess with 36m is that it's more than just the CRD count at issue.
@matthchr I'm installing it thru terraform using the helm provider. Not something special. I'm using AKS running on 1.23.8
the ability to install specific CRDs will be great! or maybe a guide on how to put this together.
I haven't tried installing ASO using the Terraform Helm provider. I've definitely done plain Helm though with no issues. Could you possibly try using vanilla Helm (possibly with the --debug flag) to try installing ASO into one of your clusters? Would help to get to the bottom of if it's Helm being slow or if it's Terraform being slow.
the ability to install specific CRDs will be great! or maybe a guide on how to put this together.
We don't have a guide for this because right now the controller will panic if it doesn't discover all of the CRDs it expects to be installed for its particular version. In order to enable this feature we'll likely need to relax this - but we haven't done that yet.
Just run a new installation on a new cluster. I was using cert-manager version 1.10, the same as the other install. using the local helm client, the installation went Ok, took nearly 2 min.
I'm starting to think that I having some throttling issues going on with AKS (private cluster) and terraform. ill try to install it on another cluster using terraform and see how it goes.
2m sounds more in the ballpark of what I would expect currently. We definitely need to do this feature, but it shouldn't be taking 36m even without it.