featureform icon indicating copy to clipboard operation
featureform copied to clipboard

[Feature Request]: add option to set custom nodeSelector values for helm deployment

Open Anntey opened this issue 1 year ago • 0 comments

Description

If you are deploying to kubernetes and have existing cluster with existing node pools, you might want to

  • A. Use pre-existing node_pool_x and assign all featureform pods to that with nodeSelector: label=node_pool_label

OR

  • B. Create new exclusive node_pool_featureform and assign all featureform pods to that pool rather than them getting randomly assigned to pre-existing node pools along with the new one.

Current Behavior

It is possible to set value for helm for nodeSelector but it isnt utilised for all services

https://github.com/search?q=repo%3Afeatureform%2Ffeatureform%20nodeselector&type=code

If I'm lacking on kube/helm knowledge and there is currently a way to set it explicitly, please advice me.

Desired Behavior

All individual services' charts utilise the value nodeSelector that is set for helm deployment

Benefits

You might not have the option to create a new kube cluster but also have existing node pools that are configured elsewhere, and you wouldn't want to use in you featureform deployment. Being able to explicitly assign pods to a node pool would help.

Possible Implementation

Add

{{- with .Values.nodeSelector }}
nodeSelector:
    {{- toYaml . | nindent 8 }}
{{- end }}

to all services' yamls

then in terraform of you deployment

resource "google_container_node_pool" "featureform_nodes" {
  ...
  node_config {
    ...
    labels = {
      env = "featureform"
    }
  }
}

resource "helm_release" "helm_featureform" {
  ...
  set {
    name = "nodeSelector.env"
    value = "featureform"
  }
}

Additional Context

Also, if it isn't obvious, in a corporate setting you can't use the featureform/terraform configurations but you have to adapt it bc you already have a VPC, kube cluster, DNS etc.

Anntey avatar Feb 07 '24 10:02 Anntey