Consider Updating CI Testing For EL To Clone Existing RAW Data

[pnadolny13]

Right now we run all EL syncs in CI with a dynamic start date of a couple days. Theres a chance that we get very little data or a subset of data thats not representative of the production data set so our tests aren't accurate. We should consider cloning the production RAW data prior to running EL syncs. This will also test any changes that will occur to the table due to singer schema updates i.e. target-snowflake will deprecate a column and create a new one of a column type has a breaking change.

Pros:

• more accurate testing of dbt because data is more robust and production-like
• EL tests cover singer schema changes and how they interact with the existing production tables

Cons:

• slower dbt run/tests because data is larger