netdoc icon indicating copy to clipboard operation
netdoc copied to clipboard

Published 20 hours ago • verified publisher icon dainok

Auto-discovery of VMs?

Open amdei opened this issue 2 years ago • 3 comments

What I have:

  1. VMWare ESXi Cluster
  2. More than dozen of hosts in that cluster for VMs
  3. Each server host more that hundred VMs
  4. Each server has up to 20 D-switches
  5. Each server has up to 10 V-switches

What I need:

  1. Discover all Clusters/switches/hosts/VMs/Interfaces/IP-Addresses/connections between VMs

How I would like to do it:

  1. Specify credentials for each cluster in netdoc
  2. Specify addresses for each cluster in netdoc
  3. Press magic button
  4. ....
  5. Profit!

What I wan as a result:

  1. Populated lists of VMs
  2. Populated lists of VM's network interfaces
  3. Populated lists of V-switches
  4. Populated lists of D-switches
  5. Populated virtual cables, connecting components to each other.
  6. Possibly, populate some auxiliary information about VM: IPs, OS type, if VMWare tools running, etc.

Why:

  1. I desperately need L2 and L3 topology of all this mess. Evolving over time and completely undocumented.

What I can:

  1. I can develop (Python, in particular)
  2. I had some experience with VMWare ESX API
  3. I tried to improve netdoc once, but was overtaken by @dainok :-) what was anyway an excellent experience!

What I'm not sure about:

  1. Idea: How my ideas fit to nebox and netdoc philosophy. Is it a right place to introduce such kind of features?
  2. Tech: Where to put all discovered entities in netbox database. I've had a hard time trying to understand what all these required fields means and how to use them properly for creating single VM...

Would it be possible for wiser guys to give a piece of advice here on these uncertainties?

The hardest part is to resolve ideological issues. Implementation should not take long, and promised to be relatively straightforward.

amdei avatar Oct 27 '22 21:10 amdei

Hello! :) I developed a similar script in the past, and if I can remember, I used pyvmomi. For sure it's interesting for netdoc, it was planned but I don't have a VMware lab right now. If you want we can work on this, I can give you my suggestions and support, but I cannot code because of the missing VMware lab. To me netdoc is useful to store network details, but we should remember that netdoc is not an asset inventory. But if I understand your needing, netbox + netdoc should be enough.

What do you think about?

Question: without NSX, ESXi do not store L3 details. So are you talking about NSX or just plain ESXi?

dainok avatar Oct 28 '22 11:10 dainok

I do not have NSX in my possession, only ESXi clusters. So, I guess L3 details could be obtained via results of previous effort - from IP address(es) on the VM. Mapping of VM NIC to OS network adapter name could be established via matching MAC addresses on both "ends".

Sure, I would like to collaborate!

But I need some guidance on both projects (netbox and netdoc) philosophy and terminology.

What I have done:

  1. Deploy latest versions of netbox and netdoc
  2. Import sample data from https://github.com/netbox-community/netbox-demo-data what allowed me to get familiar with places where to put assets details
  3. Import bunch of Device Types from https://github.com/netbox-community/devicetype-library via https://github.com/minitriga/Netbox-Device-Type-Library-Import what allowed me to make further progress of where to put other data
  4. Draw a mock-up plan of what need to be done (see below)

So far my plan looks like following: NetBox+NetDoc VMWare ESXi VMs autodiscovery

Thus, right now I have following open questions (to be updated)

  1. What netbox object should be created to represent connection from VM NIC to V-Switch? "Cable"?
  2. What would be best candidate for ESXi farm "discoverable" in netdoc? ESXi Host? Cluster? Whole Datacenter?
  3. Do we need to unconditionally create netdoc "Discoverable" for any populated VM?
  4. Do we need to unconditionally create netbox objects for V-Switches and D-Switches?
  5. Do we need to unconditionally create netbox objects for ESXi Hosts? Clusters? Datacenters?
  6. Where to put VLAN information form ESXi portgroups? NetBox/IPAM/VLANs?
  7. What about NetBox/IPAM/VLAN Groups? What are they for? Doe we need to put anything there?

amdei avatar Oct 29 '22 19:10 amdei

Let's start from the beginning: 0. I suggest to experiment with pyvmomi, once you have an idea of what you can do, it will be easier how to proceed.

  1. I guess cables, but I promise I'll give a look on that.
  2. Discoverable should be vCenters
  3. No, only vCenters and they should be manually created. There is no way to auto discover vCenters.
  4. Yes, but mind that dvSwitches are vSwitches centrally managed. From a L2 perspective dvSwitches and vSwitches are equals.
  5. Yes for ESXi Hosts, because they contains vSwitches and VMs. Clusters and datacenters are group of ESXi.
  6. Yes, VLANs.
  7. VLANs group are used by net engineers to logically group VLANs together: From a network perspective VLAN groups do not exists.

From vCenters you should retrieve:

  • ESXi hosts with main and additional IP addresses, MAC addresses, hostname, physical NICs
  • vSwitches for ESXi hosts, with attached NICs
  • VLANs
  • VMs with MAC Addresses, main and additional IP Addresses, running OS, hostname, virtual NICs
  • Clusters as groups of ESXi hosts
  • Datacenters as groups of clusters Am I missing something?

dainok avatar Oct 29 '22 20:10 dainok