Add a separate unmanaged node pool to run cluster autoscaler

[hyder]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

OKE now supports cluster autoscaler: https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengusingclusterautoscaler.htm. We need to add support for it like we have done for horizontal and vertical pod autoscalers.

New or Affected Resource(s)

Potential Terraform Configuration

# Copy-paste any Terraform configurations for how the requested feature may be used. 

References

[shapeofarchitect]

Its very critical to add the support for nodepool auto-scalar to increase the adoption for enterprise customers , I would appreciate timelines please ?

[hyder]

Hi @shapeofarchitect,

Thanks for reaching out. We have some ideas but we haven't fully tested them yet. We will probably look into this after the 4.0 release.

[mschmidt291]

Any updates on this?

[aibarbetta]

Hi @hyder, any updates on this? I'm currently using this module with the cluster autoscaler installed, the autoscaler changes generate differences in the terraform plan, is there any suggested workaround?

[hyder]

Hi @aibarbetta

I think we'll be able to start implementing this once #562 is complete. Would you be interested in testing?

[aibarbetta]

Hi @aibarbetta

I think we'll be able to start implementing this once #562 is complete. Would you be interested in testing?

@hyder sure, I can help test the changes in my environment, mention me in the PR when it's ready to test

[hyder]

After further experiments, we'll add support for cluster autoscaler gradually:

1. A separate unmanaged node pool to run cluster autoscaler (this issue)
2. IAM resources (tags, dynamic group, policies) for the worker nodes that will be running cluster autoscaler (#578)
3. Deploying cluster autoscaler itself (#579)

The unmanaged node pool needs to also be upgradable using a process similar to the current cluster and node pool upgrade process. However, we'll need to amend it somewhat.

As such, I'm renaming this issue to "add a unmanaged node pool for cluster autoscaler" instead of the full cluster autoscaler. Doing so will allow us to test each bit functionality well.

We'll also create a new dedicated milestone for adding cluster autoscaler support.

Compared to #579, I expect this issue and #578 to be done fairly quickly. #579 however will need more time. In the short term, the actual deployment of the autoscaler will be done manually. We'll have scripts and the manifests uploaded. However, users will need to modify them before creating.

Likewise, if the cluster has changed e.g. a new node pool added or removed, then the user will need to update the manifest and run apply again. In #579, we'll look at a way to automate that.

[hyder]

@aibarbetta @mschmidt291 @shapeofarchitect

[alex-old-user]

hi @hyder, not sure where is the proper place to ask, here or in #579

Is it possible just to add the following snippet to modules/oke/nodepools.tf?

  lifecycle {
    ignore_changes = [
      node_config_details[0].size
    ]
  }

Lots of ppl don't use the whole module, rather only a few submodules( modules/oke in our case), and it would solve the issue where Terraform needs to reconcile changes made by cluster-autoscaler (installed simply with Helm)

[hyder]

@devoncrouse can you please look into this?

[12345ieee]

This was affected by https://github.com/oracle-terraform-modules/terraform-oci-oke/pull/624 , before that the whole object (including size) was ignored.

The issue with adding the ignore is that it's not possible to change the size anymore after creation, effectively changing the size parameter into an initial_size.

If this is the intended way to do things, I'll look into installing the autoscaler and stop managing the size via terraform.