Task Prioritization

[wajihyassine]

With Celery/Kombu we have the capability of prioritizing which Tasks are routed to the Worker. It'd be nice to have a way to prioritize Tasks (such as FileSystemTimelineJob) to run before other Tasks and to implement some sort of priority queue (HIGH, LOW, MEDIUM).

References:

• https://docs.celeryq.dev/en/stable/userguide/routing.html
• https://docs.celeryq.dev/projects/kombu/en/stable/reference/kombu.html

One way can be to set a new field we track for each Job in Turbinia config with a PRIORITY field and that field is read and set when the Task is pushed to the queue