arthur-redshift-etl
arthur-redshift-etl copied to clipboard
Old data is (usually) better than no data for sources
In a concurrent load, extracts may be observed to fail or may stall for long enough that the load process 'times out' waiting for them.
When this happens, the last-loaded version of the data source may be available elsewhere in the warehouse (ie, if loading in staging, a copy is sitting in the non-prefixed 'standard' position). Rather than fail the relation, we should try to load our new table using the data from that existing table.