customFieldsManager <unset> casued None type has no attribute 'field'

[knight-ni]

When setting fetch_custom_attributes to False, it should not initialize allCustomAttributesNames in helpers.py which will lead to a None type has no attribute 'field' from line "for f in content.customFieldsManager.field", after I commented that code, everything work fine.

[pryorda]

@rmontenegroo Could you take a peek at this?

[rmontenegroo]

@knight-ni would paste the complete error message here? I can't make it happen here.

What is your vsphere version?

-- update -- As I said earlier, I was not able to re-create the error. I considered the possibility of the non-existence of customFieldsManager for any reason, so I made some changes to the code. I also considered the scenario where, for any crazy reason, the custom property id is not in the allCustomAttributesNames dict.

https://github.com/rmontenegroo/vmware_exporter/tree/issue_199

[der-eismann]

I have the same problem on a vSphere 6.5 host, I tried it with VSPHERE_PROD_FETCH_CUSTOM_ATTRIBUTES=True and without, no luck. This is the log output using the latest docker image:

2020-05-26 12:48:44,303 INFO:Start collecting metrics from 1.2.3.4
2020-05-26 12:48:44,303 INFO:Starting vm metrics collection
2020-05-26 12:48:44,303 INFO:Fetching vim.VirtualMachine inventory
2020-05-26 12:48:44,303 INFO:Retrieving service instance content
2020-05-26 12:48:44,304 INFO:START: _vmware_get_vm_perf_manager_metrics
2020-05-26 12:48:44,304 INFO:Fetching vim.Datastore inventory
2020-05-26 12:48:44,304 INFO:Starting host metrics collection
2020-05-26 12:48:44,304 INFO:Fetching vim.HostSystem inventory
2020-05-26 12:48:44,304 INFO:START: _vmware_get_host_perf_manager_metrics
2020-05-26 12:48:44,432 INFO:Retrieved service instance content
2020-05-26 12:48:44,563 ERROR:Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 1638, in _async_render_GET
    yield self.generate_latest_metrics(request)
twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#1, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#0, [Failure instance: Traceback: <class 'AttributeError'>: 'NoneType' object has no attribute 'field'
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:501:errback
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1475:gotResult
--- <exception caught here> ---
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib/python3.6/site-packages/twisted/python/failure.py:512:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:623:vm_inventory
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib/python3.6/site-packages/twisted/python/failure.py:512:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:476:batch_fetch_properties
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:266:<lambda>
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py:26:batch_fetch_properties
]]
--- <exception caught here> ---
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:930:vm_labels
/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py:99:parallelize
]]
--- <exception caught here> ---
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:1320:_vmware_get_vms
/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py:99:parallelize
]]

Unhandled error in Deferred:

Traceback (most recent call last):
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 930, in vm_labels
    virtual_machines, host_labels = yield parallelize(self.vm_inventory, self.host_labels)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/defer.py", line 99, in parallelize
    results = yield defer.DeferredList(args, fireOnOneErrback=True)
twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: <class 'AttributeError'>: 'NoneType' object has no attribute 'field'
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:501:errback
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:568:_startRunCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:654:_runCallbacks
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1475:gotResult
--- <exception caught here> ---
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib/python3.6/site-packages/twisted/python/failure.py:512:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:623:vm_inventory
/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py:1416:_inlineCallbacks
/usr/local/lib/python3.6/site-packages/twisted/python/failure.py:512:throwExceptionIntoGenerator
/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py:476:batch_fetch_properties
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:250:inContext
/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py:266:<lambda>
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:122:callWithContext
/usr/local/lib/python3.6/site-packages/twisted/python/context.py:85:callWithContext
/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py:26:batch_fetch_properties
]]

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 623, in vm_inventory
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 476, in batch_fetch_properties
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py", line 26, in batch_fetch_properties
    for f in content.customFieldsManager.field
builtins.AttributeError: 'NoneType' object has no attribute 'field'

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 623, in vm_inventory
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 476, in batch_fetch_properties
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py", line 26, in batch_fetch_properties
    for f in content.customFieldsManager.field
builtins.AttributeError: 'NoneType' object has no attribute 'field'

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 623, in vm_inventory
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py", line 26, in batch_fetch_properties
    for f in content.customFieldsManager.field
builtins.AttributeError: 'NoneType' object has no attribute 'field'

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 561, in host_system_inventory
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 476, in batch_fetch_properties
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py", line 26, in batch_fetch_properties
    for f in content.customFieldsManager.field
builtins.AttributeError: 'NoneType' object has no attribute 'field'

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 561, in host_system_inventory
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 476, in batch_fetch_properties
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py", line 26, in batch_fetch_properties
    for f in content.customFieldsManager.field
builtins.AttributeError: 'NoneType' object has no attribute 'field'

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 506, in datastore_inventory
    properties
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/vmware_exporter.py", line 476, in batch_fetch_properties
    properties,
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/usr/local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/usr/local/lib/python3.6/site-packages/vmware_exporter/helpers.py", line 26, in batch_fetch_properties
    for f in content.customFieldsManager.field
builtins.AttributeError: 'NoneType' object has no attribute 'field'