DAL doesn't always catch up right away

[kvanhijf]

I've managed to reproduce this issue a couple of times now, but i have no idea when it will happen. The problem is the following: I create 20 vPools simultaneously using the API. Once all vPools have status RUNNING, i delete all vPools again using the API (simultaneously). Once all vPools have been cleaned up i start over again and when creating the vPools, sometimes i don't see the created vPools in the GUI, nor does the model report they are present. However when checking the workers log, it mentions all vPools have been created successfully. After having waited a couple of minutes, suddenly the vPools appeared in the GUI and VPoolList.get_vpools also showed 20 results.

Code used to (sometimes) reproduce:

%cpaste
import json
import random
from ovs.dal.lists.albabackendlist import AlbaBackendList
from ovs.dal.lists.storagerouterlist import StorageRouterList
from ovs.dal.lists.userlist import UserList
from ovs.dal.lists.vpoollist import VPoolList
from ovs.extensions.api.client import OVSClient

user = UserList.get_user_by_username('admin')
for client in user.clients:
    if client.grant_type == 'CLIENT_CREDENTIALS':
        ovs_client = OVSClient('10.100.195.21', 443, (client.client_id, client.client_secret))
        break
else:
    raise Exception('No client found')

srs = StorageRouterList.get_storagerouters()
vpools = VPoolList.get_vpools()
backends = AlbaBackendList.get_albabackends()
for i in range(1, 21):
    sr = random.choice(srs)
    data = json.dumps({'call_parameters': {'backend_info': {'preset': 'default',
                                                            'alba_backend_guid': random.choice(backends).guid},
                                           'config_params': {'cluster_size': 4, 'dtl_mode': 'a_sync', 'dtl_transport': 'tcp', 'sco_size': 4, 'write_buffer': 128},
                                           'connection_info': {'client_id': client.client_id,
                                                               'client_secret': client.client_secret,
                                                               'host': sr.ip,
                                                               'local': True,
                                                               'port': 443},
                                           'fragment_cache_on_read': True,
                                           'fragment_cache_on_write': True,
                                           'parallelism': {'proxies': 1},
                                           'storage_ip': sr.ip,
                                           'storagerouter_ip': sr.ip,
                                           'vpool_name': 'vpool{0}'.format(i),
                                           'writecache_size': 15 }})
    ovs_client.post(api='storagerouters/' + sr.guid + '/add_vpool', data=data)
# for vpool in vpools:
#     ovs_client.post(api='vpools/' + vpool.guid + '/shrink_vpool', data=json.dumps({'storagerouter_guid': vpool.storagedrivers[0].storagerouter.guid}))
--

The bottom 2 lines are uncommented when deleting the vPools Executing on 3 node environment