netbox-sync icon indicating copy to clipboard operation
netbox-sync copied to clipboard
verified publisher icon bb-Ricardo

Wrong platform evaluation

Open lucafabbri365 opened this issue 9 months ago • 10 comments

Hello, I have a Linux virtual machine that netbox-sync reports with the wrong OS.

These are OS details from VMware:

  • OS according to the configuration file: Red Hat Enterprise Linux 7 (64-bit)
  • OS according to the VMware Tools: Red Hat Enterprise Linux 8 (64-bit)

This is vm_platform_relation specified in settings.yaml:

Red Hat Enterprise Linux = Red Hat Enterprise Linux 7, Red Hat Enterprise Linux 7.* = Red Hat Enterprise Linux 7, Red Hat Enterprise Linux 8.* = Red Hat Enterprise Linux 8

netbox-sync sets Red Hat Enterprise Linux 7 as platform instead of Red Hat Enterprise Linux 8.

This is the log content (DEBUG3 enabled):

obj.config = (vim.vm.ConfigInfo) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   changeVersion = '2025-03-13T01:25:35.881082Z',
   modified = 1970-01-01T00:00:00Z,
   name = 'LINUX01',
   guestFullName = 'Red Hat Enterprise Linux 7 (64-bit)',
   ...
   (vim.option.OptionValue) {
      dynamicType = <unset>,
      dynamicProperty = (vmodl.DynamicProperty) [],
      key = 'guestOS.detailed.data',
      value = "bitness='64' distroName='Red Hat Enterprise Linux Server' distroVersion='7.9' familyName='Linux' kernelVersion='3.10.0-1160.81.1.el7.x86_64' prettyName='Red Hat Enterprise Linux'"
      },
   ...
   (vim.option.OptionValue) {
      dynamicType = <unset>,
      dynamicProperty = (vmodl.DynamicProperty) [],
      key = 'guestInfo.detailed.data',
      value = "architecture='X86' bitness='64' cpeString='cpe:/o:redhat:enterprise_linux:8::baseos' distroAddlVersion='8.10 (Ootpa)' distroName='Red Hat Enterprise Linux' distroVersion='8.10' familyName='Linux' kernelVersion='4.18.0-553.8.1.el8_10.x86_64' prettyName='Red Hat Enterprise Linux 8.10 (Ootpa)'"
      },
   ...

obj.guest = (vim.vm.GuestInfo) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   toolsStatus = 'toolsOk',
   toolsVersionStatus = 'guestToolsUnmanaged',
   toolsVersionStatus2 = 'guestToolsUnmanaged',
   toolsRunningStatus = 'guestToolsRunning',
   toolsVersion = '12389',
   toolsInstallType = 'guestToolsTypeOpenVMTools',
   guestId = 'rhel8_64Guest',
   guestFamily = 'linuxGuest',
   guestFullName = 'Red Hat Enterprise Linux 8 (64-bit)',
   guestDetailedData = "architecture='X86' bitness='64' cpeString='cpe:/o:redhat:enterprise_linux:8::baseos' distroAddlVersion='8.10 (Ootpa)' distroName='Red Hat Enterprise Linux' distroVersion='8.10' familyName='Linux' kernelVersion='4.18.0-553.8.1.el8_10.x86_64' prettyName='Red Hat Enterprise Linux 8.10 (Ootpa)'",
   ...

As I understand, the function add_virtual_machine (connection.py) handles OS evaluation:

  1. The function first tries to extract the platform from obj.config.guestFullName - platform = "Red Hat Enterprise Linux 7 (64-bit)"
  2. If available, the function then attempts to retrieve a more accurate OS name from obj.guest.guestFullName - platform = "Red Hat Enterprise Linux 8 (64-bit)"
  3. Next, the function checks obj.config.extraConfig for the key "guestOS.detailed.data" - platform = "Red Hat Enterprise Linux"

Regarding the last point, instead of retrieving data from obj.config.extraConfig, wouldn't it be more appropriate to obtain detailed information from VMware Tools (if installed and running) via obj.guest.guestDetailedData ?

lucafabbri365 avatar Mar 17 '25 23:03 lucafabbri365

Hi,

this is very strange. The VM contains two guestInfo.detailed.data in obj.config.extraConfig?

bb-Ricardo avatar Mar 20 '25 07:03 bb-Ricardo

Hello @bb-Ricardo, the obj.config.extraConfig contains two different keys:

  1. guestOS.detailed.data 
(vim.option.OptionValue) {
      dynamicType = <unset>,
      dynamicProperty = (vmodl.DynamicProperty) [],
      key = 'guestOS.detailed.data',
      value = "bitness='64' distroName='Red Hat Enterprise Linux Server' distroVersion='7.9' familyName='Linux' kernelVersion='3.10.0-1160.81.1.el7.x86_64' prettyName='Red Hat Enterprise Linux'"
      },
  1. guestInfo.detailed.data 
(vim.option.OptionValue) {
      dynamicType = <unset>,
      dynamicProperty = (vmodl.DynamicProperty) [],
      key = 'guestInfo.detailed.data',
      value = "architecture='X86' bitness='64' cpeString='cpe:/o:redhat:enterprise_linux:8::baseos' distroAddlVersion='8.10 (Ootpa)' distroName='Red Hat Enterprise Linux' distroVersion='8.10' familyName='Linux' kernelVersion='4.18.0-553.8.1.el8_10.x86_64' prettyName='Red Hat Enterprise Linux 8.10 (Ootpa)'"
      },

Here is the code inside connection.py responsible for platform evaluation:

Image

Summary of OS Calculation Steps

  1. Primary source: config.guestFullName
  2. Fallback to actual running OS: guest.guestFullName
  3. Check extraConfig (guestOS.detailed.data) for prettyName, if available

So the prettyName comes from guestOS.detailed.data and, in my case, is equal to Red Hat Enterprise Linux leading to an "incorrect" platform evaluation (the server has Red Hat Enterprise Linux 8.10).

I suggest referring to prettyName from guestInfo.detailed.data (if available), which, if I'm not mistaken, is retrieved from VMware Tools.

According to this blog article New detailed GuestOS data in vSphere 8.0 Update 2, written by VMware person, in vSphere 8.0 Update 2 VMware enhanced the available GuestOS data using VMware Tools:

The developers wanted to make it easier for users to retrieve this information, especially if it has not changed from the last time it was collected and make this persistent so that you can quickly inventory your VMs even if it is powered off. As mentioned in the vSphere API documentation, the last known values are persisted and is available when the VM is powered off. This information is stored in the VM Advanced Setting called guestinfo.detailed.data and will contain the last known values when the VM was running.

lucafabbri365 avatar Mar 20 '25 13:03 lucafabbri365

Hi @lucafabbri365,

Thank you for doing all the research. I reworked the approach a bit. I pushed a change to the development branch. would you be able to test it?

thank you

bb-Ricardo avatar Mar 21 '25 06:03 bb-Ricardo

Hello @bb-Ricardo, Yes, it works correctly. However, I suggest one more improvement.

In some cases, depending on the installed version of VMware Tools, the prettyName field for Linux systems may not include the OS version. See the example below:

guestOS.detailed.data: bitness='64' distroName='Red Hat Enterprise Linux Server' distroVersion='7.9' familyName='Linux' kernelVersion='3.10.0-1160.81.1.el7.x86_64' prettyName='Red Hat Enterprise Linux'

guestInfo.detailed.data: bitness='64' distroName='Red Hat Enterprise Linux Server' distroVersion='7.9' familyName='Linux' kernelVersion='3.10.0-1160.114.2.el7.x86_64' prettyName='Red Hat Enterprise Linux'

netbox-sync sets the platform as Red Hat Enterprise Linux which is correct but the version is missing. I underestand this is quite specific to this particular case, but for Linux systems (familyName='Linux'), would it make sense to set the platform as distroName + " " + distroVersion ?

What's your thought ?

lucafabbri365 avatar Mar 21 '25 16:03 lucafabbri365

Hi ! Any update on this ?

Hello @bb-Ricardo, Yes, it works correctly. However, I suggest one more improvement.

In some cases, depending on the installed version of VMware Tools, the prettyName field for Linux systems may not include the OS version. See the example below:

guestOS.detailed.data: bitness='64' distroName='Red Hat Enterprise Linux Server' distroVersion='7.9' familyName='Linux' kernelVersion='3.10.0-1160.81.1.el7.x86_64' prettyName='Red Hat Enterprise Linux'

guestInfo.detailed.data: bitness='64' distroName='Red Hat Enterprise Linux Server' distroVersion='7.9' familyName='Linux' kernelVersion='3.10.0-1160.114.2.el7.x86_64' prettyName='Red Hat Enterprise Linux'

netbox-sync sets the platform as Red Hat Enterprise Linux which is correct but the version is missing. I underestand this is quite specific to this particular case, but for Linux systems (familyName='Linux'), would it make sense to set the platform as distroName + " " + distroVersion ?

What's your thought ?

rblase2 avatar Jun 16 '25 10:06 rblase2

Hi @lucafabbri365 , @rblase2 ,

I just pushed another commit. Would you be able to test it and see if this solves the linux distort version issue?

bb-Ricardo avatar Jun 17 '25 13:06 bb-Ricardo

Hi @lucafabbri365 , @rblase2 ,

I just pushed another commit. Would you be able to test it and see if this solves the linux distort version issue?

Ok, first i got error :

Traceback (most recent call last):
  File "/opt/net/netbox-sync/netbox-sync.py", line 146, in <module>
    main()
  File "/opt/net/netbox-sync/netbox-sync.py", line 111, in main
    source.apply()
  File "/opt/net/netbox-sync/module/sources/vmware/connection.py", line 399, in apply
    view_details.get("view_handler")(obj)
  File "/opt/net/netbox-sync/module/sources/vmware/connection.py", line 2195, in add_virtual_machine
    if detailed_data_dict.get("prettyName").lower() == "linux" and \
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'lower'

But I have around 1,000 virtual machines, so I tried adding this.

if detailed_data_dict.get("prettyName").lower() == "linux" and \

Instead of this :

if detailed_data_dict.get("prettyName", "").lower() == "linux" and \

And I got the correct version for all my Ubuntu machines (plus some extra info about my Windows Server — I didn’t ask for it, but I needed it, so that was perfect!). I got a second error later on with another vCenter :

Traceback (most recent call last):
  File "/opt/net/netbox-sync/netbox-sync.py", line 146, in <module>
    main()
  File "/opt/net/netbox-sync/netbox-sync.py", line 111, in main
    source.apply()
  File "/opt/net/netbox-sync/module/sources/vmware/connection.py", line 399, in apply
    view_details.get("view_handler")(obj)
  File "/opt/net/netbox-sync/module/sources/vmware/connection.py", line 2191, in add_virtual_machine
    detailed_data_key, detailed_data_value = detailed_data_item.split("=")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

And i fix it with:

        if "=" in detailed_data_item:
            detailed_data_key, detailed_data_value = detailed_data_item.split("=", 1)
            detailed_data_dict[detailed_data_key] = detailed_data_value.strip("'")
        else:
            continue

But for this one, I’m pretty sure it depends on my vCenters, not your script.

So, thanks for reacting so quickly and providing a great solution — and of course, thanks for your work on this amazing script!

rblase2 avatar Jun 17 '25 14:06 rblase2

Hey, thank you so much for testing. I was on the bus and couldn't test it. I will fix the missing parts tomorrow and let you know.

bb-Ricardo avatar Jun 17 '25 15:06 bb-Ricardo

Hi,

I just pushed another commit. Would you be able to test it? Thank you.

bb-Ricardo avatar Jun 18 '25 10:06 bb-Ricardo

Hi,

I just pushed another commit. Would you be able to test it? Thank you.

Hey! I just gave it a try — first I synced with the MAIN commit, then I tested your new commit, and everything worked perfectly, no errors at all. Now I’m going to test it with my other vCenters. If I run into any bugs, I’ll get back to you.

rblase2 avatar Jun 18 '25 10:06 rblase2