[EC2SpotManager] GCE "AttributeError: 'NoneType' object has no attribute 'startswith'"

[pyoor]

start_instances: unhandled error: Traceback (most recent call last):
File "/srv/fuzzmanager.fuzzing.mozilla.org/FuzzManager/server/ec2spotmanager/CloudProvider/CloudProvider.py", line 54, in wrapper
return wrapped(*args, **kwds)
File "/srv/fuzzmanager.fuzzing.mozilla.org/FuzzManager/server/ec2spotmanager/CloudProvider/GCECloudProvider.py", line 177, in start_instances
nodes = cluster.create(instance_type, count, conf, image=image)
File "/srv/fuzzmanager.fuzzing.mozilla.org/laniakea/laniakea/core/providers/gce/manager.py", line 266, in create
nodes = [self.gce.create_node(name, size, image, **meta)]
File "/usr/local/lib/python2.7/dist-packages/libcloud/compute/drivers/gce.py", line 4082, in create_node
size = self.ex_get_size(size, location)
File "/usr/local/lib/python2.7/dist-packages/libcloud/compute/drivers/gce.py", line 7398, in ex_get_size
zone = self.ex_get_zone(zone)
File "/usr/local/lib/python2.7/dist-packages/libcloud/compute/drivers/gce.py", line 7683, in ex_get_zone
if name.startswith('https://'):
AttributeError: 'NoneType' object has no attribute 'startswith'

[jschwartzentruber]

This error seems like a problem with the GCE API ... We get the list of zones from GCE, and then when we pass it back it fails to resolve the zone name. Updating libcloud might help if they've handled more errors like this with retries since the version we're using (2.4.0).

Otherwise, I could treat this specific error as temporary and blacklist the zone as we do for other temporary errors.