[feature] Add support for multiple Iperf servers

[Aryamanz29]

• One server is not enough for larger systems, We must specify multiple list of Iperf servers.

  • User can configure multiple iperf servers corresponding to organization

  # settings.py
    AUTO_IPERF = get_settings_value('AUTO_IPERF', True)
    IPERF_SERVERS = get_settings_value(
    'IPERF_SERVERS',
    {
        # Running on my local
        '66fe76b5-c906-4eb6-b466-6ef93492e9af': ['172.19.0.1'],
        #'<org-pk>': ['<ORG_IPERF_SERVER>']
    },)
    # In iperf.py 
    # For ex (need to make more configurable)
    servers = list(app_settings.IPERF_SERVERS.values())[0][0]

• We need to implement a lock to ensure only one Iperf check runs per server at a time.

  • Adding locking mechanism (using memcache/redis) of celery tasks that will allow running only 1 Iperf check per server. Need to ensure that cache timeout will be bigger than check celery task.

• Example :

# Using memcache
from contextlib import contextmanager
from django.core.cache import cache

@contextmanager
def memcache_lock(lock_name):
    status = cache.add(lock_name, 'lock')
    try:
        yield status
    finally:
        cache.delete(lock_name)


@task()
def iperf_check_task(info):
    with memcache_lock('lock_name') as acquired:
        do_some_work() 

• For this I've found some good references that I need to first discuss with project mentors. References :
  • https://docs.celeryq.dev/en/latest/tutorials/task-cookbook.html#ensuring-a-task-is-only-executed-one-at-a-time
  • https://stackoverflow.com/questions/12003221/celery-task-schedule-ensuring-a-task-is-only-executed-one-at-a-time

[pandafy]

We need to implement a lock to ensure only one Iperf3 check runs per server at a time.

• Adding locking mechanism (using memcache/redis) of celery tasks that will allow running only 1 Iperf3 check per server. Need to ensure that cache timeout will be bigger than check celery task.

As a secondary goal, we should allow running checks on different servers at the same time. E.g. if there are two iperf3 server, then one check can be run on each of them at a given time.