Develop an HA plan for the batch submitter service

[CLEckhardt]

Develop a plan and design doc for a high-availability deployment of the batch submitter service.

To do

• [x] Determine overall HA approach
• [x] Determine and demonstrate that we can/cannot safely make atomic updates with postgres (i.e. can implement a concurrency-safe job queue with locks on service_ids)
• [x] Determine and demonstrate that we do/do not need some form of down detection
• [x] Write up concurrency options and tradeoffs (determine database load)
• [x] Map out the queries and data model related to instance coordination
• [x] Determine the impact of using SQLite in an HA configuration
• [x] Map out the impact of an HA configuration to the queuer, submitter, and DLT monitor