[FEAT] - Externalize Config - Max Merge Days Overrides for Each Module

[GeekSheikh]

Currently, job runs exceeding 30d will never receive their final closure and any runtime after 30d will be disregarded. This is very acceptable for most customers but some customers have long-running streams that exceed this limit.

To accommodate this a customer should be able to override the default 30d maximum in the Overwatch config. This will come at the cost of minor performance degradation but will be required for customers with jobs frequently exceeding 30d.

The 30 in the incremental DF load needs to be variable as per customer overrides https://github.com/databrickslabs/overwatch/blob/e2193e7e5c8b5227a102d00c906a4c15c314ac33/src/main/scala/com/databricks/labs/overwatch/pipeline/Silver.scala#L237

[GeekSheikh]

Initially this was thought to be for only jobrun but overrides for all lagging merges should be externalized as possible overrides for customer situations. Examples include CLSF, JR, Spark, etc.

[GeekSheikh]

More thought needs to go into how to do this. 30d is already quite a bit for some customers and 99% of jobs finish within 24h...so it's a LOT of extra compute that can probably be removed a little more clever approach.