data-infra
data-infra copied to clipboard
Deprecate obsolete `staging` tables/views
As an analytics engineer, I want to deprecate outdated & unused tables and views in our data warehouse to reduce the risk of users accidentally using incorrect or obsolete data and to reduce noise and clutter in audit activities (for example, auditing for models lacking documentation.)
AC:
- [ ] Investigate all models in the
staging
dataset that aren't being updated regularly (see query below)- [ ] Check whether they appear in the dbt project (some of these are incremental models)
- [ ] Check whether there is documentation of their intended purpose
- [ ] Deprecate unused
staging
models
-- query to identify outdated models
SELECT
table_name, table_type, creation_time
FROM
`cal-itp-data-infra`.`staging`.INFORMATION_SCHEMA.TABLES
WHERE
creation_time < "<yesterday's date>"
ORDER BY creation_time