exporterhub.io icon indicating copy to clipboard operation
exporterhub.io copied to clipboard

Scheduler not working when the CSV update

Open ralfyang opened this issue 3 years ago • 1 comments

Scheduler has some problem(not working) after the CSV file has been updated in the exporterhub.io

ralfyang avatar Aug 20 '21 01:08 ralfyang

I have tested scheduler job create_or_update_exporters on csv file change by setting it to run every minute and it seems working well.

Steps to Reproduce

  1. set create_or_update_exporters to run every minute in api/hub/management/commands/scheduler.py
def listener(event):
    if not event.exception:
        if scheduler.get_job('check_token'):
            job = scheduler.get_job('check_token')
       
            if job.func() == 'VALID_TOKEN':
                scheduler.remove_job('check_token')
                logger.info('Remove check_token job.')

                scheduler.add_job(
                    create_or_update_exporters,
                    #trigger            = CronTrigger(hour='*/4'),
                    trigger            = CronTrigger(minute='*'),
                    id                 = 'create_or_update_exporters',
                    max_instances      = 1,
                    replace_existing   = True,
                    coalesce           = True,
                    misfire_grace_time = 900,
                    next_run_time      = datetime.now()
                )
                logger.info("Added job 'create_or_update_exporters'.")
  1. Run all containers
make build && make run
  1. (other terminal) check logs from scheduler container
docker-compose logs -f scheduler
# create_or_update_exporters job is running every hour
scheduler_1   | CHECK_EXPORTERS_DONE
scheduler_1   | CHECK_EXPORTERS_START
scheduler_1   | CHECK_EXPORTERS_DONE
scheduler_1   | CHECK_EXPORTERS_START
  1. SSH into docker scheduler container and append new exporter in /data/exporter_list.csv
docker-compose exec scheduler /bin/bash
root@032832075fb2:/data# echo "Traefik Exporter,traefik-exporter,https://github.com/iadvize/traefik-exporter,0,HTTP" >> /data/exporter_list.csv
  1. Check logs from scheduler container and see if new exporter is added to the database
# ...
scheduler_1   | CHECK_EXPORTERS_START
scheduler_1   | CHECK_EXPORTERS_DONE
scheduler_1   | CHECK_EXPORTERS_START
scheduler_1   | id: 42 name: traefik-exporter | SUCCESSFULLY_ADDED_REPOSITORY_AND_RELEASES | 2021-09-04 00:22:57.102681
scheduler_1   | CHECK_EXPORTERS_DONE
scheduler_1   | CHECK_EXPORTERS_START
scheduler_1   | CHECK_EXPORTERS_DONE
# ..

bart2001 avatar Sep 03 '21 15:09 bart2001