NoneType' object has no attribute 'items'

[gthomson31]

Hi There

Stumbled upon this issue when building out a flask application but have also noticed this when just testing the code in a simple consistent loop.

Looking for pointers if there is something obvious I am missing in this.

Using Username/Password to authenticate.

Error usually occurs after 10-15 mins of running without issue.

SCRIPT CODE

foundation_name = 'lab'
target_endpoint = '<URL>'

proxy = dict(http=os.environ.get('HTTP_PROXY', ''), https=os.environ.get('HTTPS_PROXY', ''))
client = CloudFoundryClient(target_endpoint, proxy=proxy, verify=False)

client.init_with_user_credentials('<USERNAME>','<PAASWORD>')

def app_buildpacks():
	app_buildpack_usage = []
	app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename' , 'Locked']

	
	for app in client.v2.apps:
		app_name = app['entity']['name']
		app_buildpack_name = app['entity']['detected_buildpack'] or app['entity']['buildpack'] 
		app_buildpack_guid = app['entity']['detected_buildpack_guid']


		for buildpack in client.v2.buildpacks:

			buildpack_guid = buildpack['metadata']['guid']

			if app_buildpack_guid == buildpack_guid:

				buildpack_name = buildpack['entity']['name']
				buildpack_stack = buildpack['entity']['stack']
				buildpack_filename = buildpack['entity']['filename']
				buildpack_lock = buildpack['entity']['locked']

				d = { 
				'app_name' : app_name , 
				'buildpack_name' : buildpack_name,
				'buildpack_stack' : buildpack_stack , 
				'buildpack_guid' : buildpack_guid, 
				'buildpack_filename': buildpack_filename,
				'buildpack_lock' : buildpack_lock
				}

				app_buildpack_usage.append(d)

	# Sort the list by name 
	app_buildpack_usage = sorted(app_buildpack_usage, key = lambda i: i['buildpack_name'])
	
	# Return the two values ( Builpack usage list of dicts and table headers )
	return app_buildpack_usage,app_buildpack_table_field_names


def background():
  global app_buildpack_usage, app_buildpack_table_field_names

  while True:
    try:
        app_buildpack_usage,app_buildpack_table_field_names = app_buildpacks()



def foreground():
  @app.route('/app_buildpacks')
  def app_buildpacks_page():
    return render_template('app_buildpack_table.html', colnames=app_buildpack_table_field_names, app_buildpack_usage=app_buildpack_usage)



b = threading.Thread(name='background', target=background)
f = threading.Thread(name='foreground', target=foreground)


b.start()
f.start()

ERROR

  \Python37\lib\site-packages\cloudfoundry_client\v2\entities.py", line 66, in _list
    yield entity_builder(list(resource.items()))

  \Python37\lib\site-packages\cloudfoundry_client\v2\apps.py", line 51, in <lambda>
    lambda pairs: _Application(target_endpoint, client, pairs))

  \Python37\lib\site-packages\cloudfoundry_client\v2\entities.py", line 17, in __init__
    for attribute, value in list(self['entity'].items()):

AttributeError: 'NoneType' object has no attribute 'items'

[antechrestos]

Hi,

I've been running the following code and it works nice... :cry:

import logging
import os
import threading

from cloudfoundry_client.client import CloudFoundryClient
from flask import jsonify, Flask

logging.basicConfig(level=logging.INFO,
                    format='%(levelname)5s - %(name)s -  %(message)s')

foundation_name = 'lab'
target_endpoint = 'https://somewhere.org'

proxy = dict(http=os.environ.get('HTTP_PROXY', ''), https=os.environ.get('HTTPS_PROXY', ''))
client = CloudFoundryClient(target_endpoint, proxy=proxy, verify=False)

client.init_with_user_credentials('user', 'password')

app_buildpack_usage = []
app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename', 'Locked']

def app_buildpacks():
    app_buildpack_usage = []
    app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename', 'Locked']

    for app in client.v2.apps:
        app_name = app['entity']['name']
        app_buildpack_name = app['entity']['detected_buildpack'] or app['entity']['buildpack']
        app_buildpack_guid = app['entity']['detected_buildpack_guid']

        for buildpack in client.v2.buildpacks:

            buildpack_guid = buildpack['metadata']['guid']

            if app_buildpack_guid == buildpack_guid:
                buildpack_name = buildpack['entity']['name']
                buildpack_stack = buildpack['entity']['stack']
                buildpack_filename = buildpack['entity']['filename']
                buildpack_lock = buildpack['entity']['locked']

                d = {
                    'app_name': app_name,
                    'buildpack_name': buildpack_name,
                    'buildpack_stack': buildpack_stack,
                    'buildpack_guid': buildpack_guid,
                    'buildpack_filename': buildpack_filename,
                    'buildpack_lock': buildpack_lock
                }

                app_buildpack_usage.append(d)

    # Sort the list by name
    app_buildpack_usage = sorted(app_buildpack_usage, key=lambda i: i['buildpack_name'])

    # Return the two values ( Builpack usage list of dicts and table headers )
    return app_buildpack_usage, app_buildpack_table_field_names


def background():
    global app_buildpack_usage, app_buildpack_table_field_names

    while True:
        try:
            app_buildpack_usage, app_buildpack_table_field_names = app_buildpacks()
            logging.info("App buildpacks refreshed")
        except Exception as ex:
            logging.exception("Exception while getting buildpacks")


def foreground():
    global foundation_name
    app = Flask(foundation_name)

    @app.route('/app_buildpacks')
    def app_buildpacks_page():
        # return render_template('app_buildpack_table.html', colnames=app_buildpack_table_field_names,
        #                        app_buildpack_usage=app_buildpack_usage)
        return jsonify(app_buildpack_usage)

    app.run(port=os.environ.get('PORT', 9010))


b = threading.Thread(name='background', target=background)
f = threading.Thread(name='foreground', target=foreground)

b.start()
f.run()

I will publish a version that improve the information got from error and will let you use it to better en light it

Thanks

[antechrestos]

For infomormation I also rewrote a little your code as follows to optimize api call:

import logging
import os
import threading

from cloudfoundry_client.client import CloudFoundryClient
from flask import Flask, jsonify

_logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO,
                    format='%(levelname)5s - %(name)s -  %(message)s')

target_endpoint = 'https://somewhere.org'

proxy = dict(http=os.environ.get('HTTP_PROXY', ''), https=os.environ.get('HTTPS_PROXY', ''))
client = CloudFoundryClient(target_endpoint, proxy=proxy, verify=False)

app = Flask(__name__)

app_buildpack_usage = []

app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename', 'Locked']


def app_buildpacks():
    global _logger

    next_app_buildpack_usage = []

    buildpacks_by_guid = {buildpack['metadata']['guid']: buildpack for buildpack in client.v2.buildpacks}

    for app in client.v2.apps:
        app_name = app['entity']['name']
        app_buildpack_guid = app['entity']['detected_buildpack_guid']
        app_buildpack = buildpacks_by_guid.get(app_buildpack_guid)
        if app_buildpack is not None:
            buildpack_name = app_buildpack['entity']['name']
            buildpack_stack = app_buildpack['entity']['stack']
            buildpack_filename = app_buildpack['entity']['filename']
            buildpack_lock = app_buildpack['entity']['locked']

            d = {
                'app_name': app_name,
                'buildpack_name': buildpack_name,
                'buildpack_stack': buildpack_stack,
                'buildpack_guid': app_buildpack_guid,
                'buildpack_filename': buildpack_filename,
                'buildpack_lock': buildpack_lock
            }

            next_app_buildpack_usage.append(d)

    # Sort the list by name
    return sorted(next_app_buildpack_usage, key=lambda i: i['buildpack_name'])


def background():
    global app_buildpack_usage, _logger

    while True:
        try:
            app_buildpack_usage = app_buildpacks()
            _logger.info('App buildpacks refreshed')
        except Exception as _:
            _logger.exception("Error while refreshing buildpacks")
            return


@app.route('/app_buildpacks')
def app_buildpacks_page():
    global app_buildpack_table_field_names, app_buildpack_usage
    # return render_template('app_buildpack_table.html', colnames=app_buildpack_table_field_names,
    #                        app_buildpack_usage=app_buildpack_usage)
    return jsonify(app_buildpack_usage)


background_thread = threading.Thread(name='background', target=background)
if __name__ == '__main__':
    client.init_with_user_credentials('user', 'password')
    background_thread.start()
    app.run(port=os.environ.get('PORT', 9000))

It may be good to add a sleepin your background so as not to flaod your endpoint

[antechrestos]

@gthomson31 I've just released the version 1.9.0 that will raise a dedicated exception InvalidEntityException containing the raw entity. Can you please run your code with this version and send me the error.

Thanks

[gthomson31]

Cheers will give it a go

[kcheng555]

Hello there, Has any progress been made on this? Also, I'm using version 1.9.0 it seems that this issue is still raising an AttributeError.

  File "/usr/lib/python3.7/site-packages/cloudfoundry_client/v2/entities.py", line 70, in _list
     yield entity_builder(list(resource.items()))
  File "/usr/lib/python3.7/site-packages/cloudfoundry_client/v2/apps.py", line 51, in <lambda>
     lambda pairs: _Application(target_endpoint, client, pairs))
  File "/usr/lib/python3.7/site-packages/cloudfoundry_client/v2/entities.py", line 21, in __init__
     for attribute, value in list(self['entity'].items()):
AttributeError: 'NoneType' object has no attribute 'items'

[DerCoop]

Hey kcheng55, thx for the report. please update your cloudfoundry client to the 1.10.

are your sure that you use the code of 1.09? The exception was fixed exactly with this version but your code looks like the older one.

[kcheng555]

I'm working on updating to 1.10.0 to see if that will fix the issue, but I'm not sure how to reproduce it, so it's a waiting game.

I've verified that I'm using version 1.9.0 by printing cloudfoundry_client.__version__. It looks like my stack trace has slightly different line numbers than the one in the OP, which seem to line up with the changes made here : https://github.com/cloudfoundry-community/cf-python-client/commit/435d1090fc1a55971ad40c94cdbc9978f0f835e5

[antechrestos]

That seems to be related to entity key being present in json object yet valued to null I am currently on holiday without any laptop

[DerCoop]

@antechrestos yes, but you fixed (add a check) it with the 1.9 ;) So, happy hollliday ;)

[antechrestos]

Partially it seems. I though the entity was not present. In fact it is but valued to null. If that is the case, what to do with it?

[DerCoop]

null object is, like no object, not parseable. In both cases we will raise an invalid entity so

extend the test to check if the field exists and is not null something like if self.get('entity') is not None

[antechrestos]

Agreed