Consider Splitting Update and Process Jobs

[boyter]

Its a little complex that both are contained in the same code. Splitting them apart would have some testing benefits. Since the checks to see what files should be processed are based on file update times there is no need for the results of one to be coupled to the next anymore.

We would need to have a number of threads for updates feeding finished results into a queue to be processed by a number of processing threads.

The only downside I can think of currently is that we would be spawning additional threads and would need to update the number of quartz jobs to compensate.