[ENH] Use k8s manifests/kustomize to deploy dask-gateway

[marcelovilla]

https://github.com/nebari-dev/nebari/pull/2609 was recently merged and it includes the logic to deploy k8s manifests and Helm charts directly instead of doing it via Terraform. I think we should build on top of that approach to deploy the dask-gateway Helm chart.

Originally posted by @marcelovilla in #2620

The goal for this issue is to refactor this stage: https://github.com/nebari-dev/nebari/tree/main/src/_nebari/stages/kubernetes_services/template/modules/kubernetes/services/dask-gateway and move all the configurations that we are doing right now through terraform resources, and instead use the available dask-gateway helm chart

How we would deploy this service will also change; it will have its own service and must follow a very similar structure as https://github.com/nebari-dev/nebari/tree/main/src/_nebari/stages/kubernetes_kuberhealthy

For whoever picks this issue, I would start familiarizing myself with how the kuberhealthy stage works, and then make a copy of it for the dask-gateway service, and start by replacing the associated yaml values based on what we are doing here https://github.com/nebari-dev/nebari/blob/d680ca8294b9f29e38dbffb93a2c7f5e9188393d/src/_nebari/stages/kubernetes_services/template/modules/kubernetes/services/dask-gateway/gateway.tf#L113-L154

Note that a few resources, like PVC/PV/Secrets and others, might already be handled by the helm chart, so you will need to address the gaps accordingly.

[Helion55]

I want to start working on this issue, @marcelovilla can you guide me with a breif overview about the status of this now.

[viniciusdc]

Hey @Helion55 I've updated the issue description, let me know if you have any questions