hashicorp
Add support for supervisor and guest namespaces
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Description
I'd like to propose the ability to manage vSphere supervisor (and guest) namespaces as a new resource.
Potential Terraform Configuration
A resource like:
resource "vsphere_supervisor_namespace"
that might be options from the namespace_management APIs to configure supervisor namespaces, and a similar/separate resource for guest namespaces for namespace APIs
References
I did not see any issues that might point to something similar, but the context of my request is related to namespaces as described here, for clarity: https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-kubernetes/GUID-177C23C4-ED81-4ADD-89A2-61654C18201B.html
Looks like the initial code for supporting this has been added to govmomi.
https://github.com/vmware/govmomi/pull/2143
There is no support for guest namespaces yet.
@rickhlx @jmarhee @bill-rich is there any traction on this? It would be really awesome to have this provider also support tanzu functionality, this would allow an almost end to end deployment of a working tanzu kubernetes cluster via terraform which could be a huge time saver for people wanting to set this up.
My deployments in a home lab so I opted for HAProxy > NSX-T since the resource demands were much slimmer. Feel free to borrow, modify and mangle my helper module I created which parses out parameters with minimal input: https://github.com/jd4883/terraform-supervisory-cluster-parameter-parser-vsphere | https://registry.terraform.io/modules/jd4883/cluster-parameter-parser-vsphere/supervisory. NOTE: as I do not use NSX-T the parameters were planned around HAProxy and this is not all inclusive of the potential options the API supports, but it should be a nice time saver for parameterizing what would be needed for a fresh supervisory cluster
I've personally been doing the supervisor cluster + namespace part by hand but did find that there are API endpoints to do this using the VMware API explorer. The relevant endpoints I found which I presume govmomi may already support (from reading the PR linked above) would be the following:
- /api/vcenter/namespace-management/clusters/${compute_cluster_id}?action=enable
- /api/vcenter/namespace-management/clusters/${compute_cluster_id}?action=disable
I did not yet try creating namespaces via API once the supervisory cluster was created but it seems like it may either be configurable within the same API endpoints or some of the related ones; one thing I will not is that the payload for creating a supervisory cluster does want a parameter, workload_portgroup, which as I understand it is an ID from a portgroup which can be created as a resource in terraform, but not looked up as a data block (this could be very useful instead of needing to provide a static ID or relating a resource block to it). All other parameters the API docs indicate you need I was able to derive with terraform data lookups.
Very curious and happy to test out an updated vsphere provider that has support for namespace management, hope to see this in a future release
Hi @jd4883 ~ for the vSphere provider it still remains in the backlog as can be viewed for the attached milestone.
In the meantime, please refer to this tweet by Adam Fowler.
Ryan Johnson Staff II Solutions Architect | VMware, Inc.
FYI This feature has been implemented by myself in an upcoming (not yet a public repo... watch this space over the coming hours/days) provider: https://github.com/vmware-tanzu-labs/terraform-provider-namespace-management
There's a few changes required in upstream projects (govmomi) for this provider so you won't find it in the provider registry yet, but we plan on working fast towards a release with basic functionality. Tested on vSphere 7.0u2 and 7.0u3 with vSphere networking and Avi. Will also support NSX-T networking. Full details on the above link when it becomes public.
Edit: I should mention the new repo also includes an initial terraform module that does the end to end - install Avi (using the Avi provider), configure Avi for Tanzu (Avi provider again), and enable the supervisor cluster (using the namespace-management provider). In future that module will have options for creating the switches / portgroups too. (Using this vsphere provider). This module will eventually be in its own repo as per the Terraform Module recommendations.
Adam Fowler Advisory Platform Architect, Modern Applications (Tanzu) team | VMware, Inc.
Hi @adamfowleruk, this is great news! When this does move forward, will this support HAProxy as a Load Balancer as well? I have a vpshere setup that works pretty well right now that I would ideally like to back by code so a provider would be awesome long-term, look forward to putting your provider to use. Let me know if there is any testing / validation that would be helpful and I'd be happy to provide this.
Cheers,
Jacob
This provider is now public! Only in Alpha stage so far (so not listed on the Hashicorp registry) but we hope to get it there by beta-1. See the README for rapid dev plans.
https://github.com/vmware-tanzu-labs/terraform-provider-namespace-management
@jmarhee FYI!
@jd4883 I don't think we'll support HA-Proxy, as that's been removed from the support list of TKG itself I believe since 7.0u2, but feel free to open an issue on the new repo with any and all enhancement requests you want! (A rough idea of relative priority would be greatly appreciated!). Also feel free to add in any work you already have to provider or modules as PRs (preferably with an issue logged first - GitFlow approach). It's all Apache-2.0 and the only admin is agreement with the VMware CLA which isn't onerous.
Adam Fowler Advisory Platform Architect, Modern Applications (Tanzu) team | VMware, Inc.
@adamfowleruk thanks for the update! I am excited to try this out; my guess is within the next week I'll get a chance to fiddle with this and my deployment.
Can you provide documentation on HAproxy support going away? As I have not seen this and I am deifnitely running on 7.0u3 without issues. I believe support for HAProxy instead of exclusively NSX-T came around in 2021. For a lab environment, the VM is extremely lightweight and has minimal drawbacks from what I've seen. I tried using the VCF deploy method when tanzu was first announced and found that ate a bit too much of my hardware on my home server to work (originally NSX-t was requiring BGP as well, though I think static routing is valid now too). I've had no issues in my home lab with the haproxy node having any sort of performance issues so if it is officially vmware supported and will continue to be, I would presume others would appreciate the lighter weight hardware requirements for setting up tanzu in a homelab type environment.
I have terraform code to spin up the HAProxy VM as well (if i remember right its 100% hands off). I did a bit of checking out the go code and may understand it well enough to make suggestions for support for haproxy as a load balancer; I'm pretty sure if I hook up the provider and get it working in my setup I'll have a pretty good idea of what field diffs are in place if I am able to import, so I'll provide any insights I have here once I get a chance to focus on this. Any contributions I'll try to follow the gitflow approach with an issue and PRs as requested. This is a hobby for me so its not my top priority with time but it would be really awesome to contribute towards something that can help others setup tanzu via terraform.
Looking at your examples I have a lot of similarly handled pieces; though I keep the different cluster components in separate workspaces for the moment (I found it easier to create reliably without worry about cleaning too many items at once, plus it keeps the moving parts more manageable). I'll see to it that if there are some helpful PR's I can include I will do so.
Thanks for putting this together and look forward to seeing how it evolves!