vcenter-netbox-sync
vcenter-netbox-sync copied to clipboard
synackray
Plugin dont work after update netbox to 2.9.1
Environment
Python version: 3.6.8 NetBox version: 2.9.1 vCenter version: 6.7.0 -->
- vCenter NetBox Sync version:
Steps to Reproduce
- After run plugin i see
[root@netbox]# /bin/python3.6 /opt/vcenter-netbox-sync/run.py >/var/log/vcentersync.log
2020-08-23 15:56:17,690 [INFO] Created new HTTP Session for NetBox.
2020-08-23 15:56:17,785 [INFO] Detected NetBox API v2.9.
2020-08-23 15:56:17,786 [INFO] Verifying all prerequisite objects exist in NetBox.
2020-08-23 15:56:17,838 [INFO] NetBox manufacturers object 'VMware' match current values. Moving on.
2020-08-23 15:56:17,866 [INFO] NetBox platforms object 'VMware ESXi' match current values. Moving on.
2020-08-23 15:56:17,895 [INFO] NetBox platforms object 'Windows' match current values. Moving on.
2020-08-23 15:56:17,919 [INFO] NetBox platforms object 'Linux' match current values. Moving on.
2020-08-23 15:56:18,032 [INFO] NetBox sites object 'vCenter' do not match current values.
Traceback (most recent call last):
File "/opt/vcenter-netbox-sync/run.py", line 1601, in
Expected Behavior
Observed Behavior
Same here, even after a cleanup, collect seem to work and stop with unexpected behavior.
also it seem not working properly with vcenter/esxi 7.0 which have been updated on API level.
Probably related to https://github.com/netbox-community/netbox/issues/3703
Can you paste a debug run log + results of GET /api/dcim/sites/?name=vCenter?
2020-08-24 10:00:02,237 [INFO] Verifying all prerequisite objects exist in NetBox. 2020-08-24 10:00:02,268 [INFO] NetBox manufacturers object 'VMware' match current values. Moving on. 2020-08-24 10:00:02,304 [INFO] NetBox platforms object 'VMware ESXi' match current values. Moving on. 2020-08-24 10:00:02,329 [INFO] NetBox platforms object 'Windows' match current values. Moving on. 2020-08-24 10:00:02,358 [INFO] NetBox platforms object 'Linux' match current values. Moving on. 2020-08-24 10:00:02,426 [INFO] NetBox sites object 'vCenter' do not match current values. 2020-08-24 12:00:01,821 [INFO] Created new HTTP Session for NetBox. 2020-08-24 12:00:01,866 [INFO] Detected NetBox API v2.9. 2020-08-24 12:00:01,868 [INFO] Verifying all prerequisite objects exist in NetBox. 2020-08-24 12:00:01,898 [INFO] NetBox manufacturers object 'VMware' match current values. Moving on. 2020-08-24 12:00:01,932 [INFO] NetBox platforms object 'VMware ESXi' match current values. Moving on. 2020-08-24 12:00:01,969 [INFO] NetBox platforms object 'Windows' match current values. Moving on. 2020-08-24 12:00:01,999 [INFO] NetBox platforms object 'Linux' match current values. Moving on. 2020-08-24 12:00:02,069 [INFO] NetBox sites object 'vCenter' do not match current values.
`HTTP 200 OK Allow: GET, POST, HEAD, OPTIONS Content-Type: application/json Vary: Accept
{ "count": 1, "next": null, "previous": null, "results": [ { "id": 12, "url": "http://netbox..com/api/dcim/sites/12/", "name": "vCenter", "slug": "vcenter", "status": { "value": "active", "label": "Active" }, "region": null, "tenant": null, "facility": "", "asn": null, "time_zone": null, "description": "", "physical_address": "", "shipping_address": "", "latitude": null, "longitude": null, "contact_name": "", "contact_phone": "", "contact_email": "", "comments": "A default virtual site created to house objects that have been synced from vCenter.", "tags": [ { "id": 10, "url": "http://netbox..com/api/extras/tags/10/", "name": "Synced", "slug": "synced", "color": "9e9e9e" }, { "id": 9, "url": "http://netbox.**.com/api/extras/tags/9/", "name": "vCenter", "slug": "vcenter", "color": "9e9e9e" } ], "custom_fields": {}, "created": "2020-05-13", "last_updated": "2020-05-13T20:34:40.272068+03:00", "circuit_count": null, "device_count": null, "prefix_count": null, "rack_count": null, "virtualmachine_count": null, "vlan_count": null } ] }`
Yes so this script is incompatible with Netbox >= 2.9.0 due to changes in the way tags are created. Best to stay on 2.8 until a PR is submitted by someone with the necessary changes.
Not just the tags, but also the API changes in 2.9.0 for ipam.IPAddress cause the mapping between IP & virtual-machine to not work anymore.
ipam.IPAddress: Removed interface field; replaced with assigned_object generic foreign key. This may represent either a device interface or a virtual machine interface. Assign an object by setting assigned_object_type and assigned_object_id.
this will fix the tag issue: https://github.com/bb-Ricardo/vcenter-netbox-sync/commit/2d3052fd14bc3947b24e1da3927e93e4e44b6355
Will have a look at ip ipam/ipaddress next week.
Well, we got some progress: https://github.com/bb-Ricardo/vcenter-netbox-sync/commit/9447109a12e18703954cd843c7bc2a6adb936285
Now all my items sync again as before.
Please check it out. All changes are also in #132
I have a problem after updating the plug-in, all virtual servers were transferred to Standalone ESXi Host, they were previously synchronized with their server, for example, if the virtual machine is on the 192.168.50.170 server, then in the Cluster it will also be synchronized there. Now all VMs are in Standalone ESXi Host.
also, as you can see in the screenshot, some virtual servers are in their clusters, but the main part has moved to Standalone
That's very interesting as I didn't touch this part of code: https://github.com/bb-Ricardo/vcenter-netbox-sync/blame/9447109a12e18703954cd843c7bc2a6adb936285/run.py#L557
"Standalone ESXi Host" is used if the hosts cluster name is the same as the hosts name:
cluster = obj.parent.name
# We throw the cluster away if it matches the ESXi host
# name as it is standalone.
if cluster == obj_name:
# Store the host so that we can check VMs against it
self.standalone_hosts.append(cluster)
cluster = "Standalone ESXi Host"
and Line: https://github.com/bb-Ricardo/vcenter-netbox-sync/blob/develop/run.py#L647
elif vc_obj_type == "virtual_machines":
# Virtual Machines
log.debug(
"Collecting info for virtual machine '%s'", obj_name
)
# Cluster
cluster = obj.runtime.host.parent.name
if cluster in self.standalone_hosts:
log.debug(
"VM is assigned to a standalone ESXi host. Setting "
"cluster to 'Standalone ESXi Host'."
)
cluster = "Standalone ESXi Host"
In my opinion the whole sync needs a complete rewrite. Unfortunately this code is only used in a test environment. It was a POC and after the last NetBox API FU I'm not sure if I'm willing to spend my time on changing code and adapting APIs every time a new NetBox major release is issued.
I have loads of ideas but (https://github.com/bb-Ricardo/vcenter-netbox-sync/issues) but not sure if time allows it.
@bb-Ricardo I keep getting errors like this: [DEBUG] NetBox 400 status reason: {"tags":[["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: Synced"],["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: vCenter"]]}
I am on your develop branch
@MarcSN311 Not sure if it the exact same problem I ran into, but I had to delete the vCenter and Synced tag before running it. I am running 2.9.4
I am on a fresh install, I did not have any tags before running the sync.
EDIT: Nevermind, I found the issue on my end.
@fjasboka
do you have a problem with clusters? like mine https://github.com/synackray/vcenter-netbox-sync/issues/129#issuecomment-698996038
or don't you use them?
@SerhiiZahuba I did not run into that issue. It placed all the VMs in the correct cluster and also the ESXi servers.
Strange, I don’t understand what I’m doing wrong. All my virtual machines are migrated to the Standalone ESXi Host section.
@bb-Ricardo I keep getting errors like this:
[DEBUG] NetBox 400 status reason: {"tags":[["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: Synced"],["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: vCenter"]]}I am on your develop branch
Any progress on this issue?
currently trying to rewrite this whole process.
Hope I get all of this covered in here: https://github.com/bb-Ricardo/netbox-sync
I went down a rabbit hole and finally can see the light at the end of the tunnel again. This took way longer then expected (as usual).
Here you go: https://github.com/bb-Ricardo/netbox-sync
please test as much as you can: Debug logs are very welcome.
Just gave it a first try and looks very promising! Only thing I've noticed so far is my vlans in vmware are named "123-name-10.0.0.0/24" and they get imported into netbox with the url encoded %2f instead of a / (when looking at the name column on the VM interface)
Great to hear.
Can you please open an issue for that. Would like to have a look at it.
Thank you.