Add an ability to give order the paths in Grabbit Configs

[sagarsane]

There should be some way of ordering paths in a grabbit config.

This could be manual/user driven (something like order:1, order:2 for each path in the config) or automated / programmatically calculated in some way (by looking at the paths in question and figuring out which paths should go in first)

[jdigger]

This is to address the sequencing issue caused by concurrency and the issue described in GH-92.

[jbornemann]

A potential solution pasted from https://github.com/TWCable/grabbit/pull/144:

1.) We need to create a ranking for paths. Here are some example paths for demonstration:

/foo - 1 /foo/bar - 2 /foo/bar/zoo - 3 /foo/car/do - 1

Start each path out with a ranking of 1, then for each path in sorted order based on path depth, increment the path's ranking by it's nearest parent's ranking if it is found within the visited set; otherwise keep a ranking of 1.

This makes sure paths that can be processed independently are within the same rank group.

After that, we need to make sure Spring Batch will feed the jobs from our job queue, into the thread pool in order, based on ranking. To do this, we would need a Spring ThreadPoolTaskExecutor instance with a ThreadPoolExecutor that is built with a BlockingPriorityQueue implemented with a Comparator that compares our jobs based on path ranking. Long winded, I know..

Still even then, this would only make sure that jobs are feed into the pool in priority order, but it wouldn't prevent certain sized thread pools from picking up and starting work on higher ranked jobs before lower ranked jobs complete. To solve this, we would probably need some sort of system within our validator task that polls the job queue to see if lower ranked jobs are still in flight, and if so, put the thread of the higher ranked job to sleep for some period.

[Shashi-Bhushan]

Had a discussion with @jbornemann regarding this, quoting him from the discussion

Any solution will be too complicated compared to the value it would give us. At the very minimum, what we have now (checking to see if path already has an existing parent before running the path's job - and raising a configuration error) I think is acceptable.

Closed the PR i created couple of weeks back. Should this ticket be closed as well, or we keep it in To-Do list for a later date. @jdigger @jazeren1 @sagarsane @viveksachdeva

[sagarsane]

Thanks @Shashi-Bhushan .. Yes I think we should keep this open for now.