bdit_data-sources
bdit_data-sources copied to clipboard
Reform process for creating unique IDs for Open Data
For each api_id in the wys.locations table, the record with the earliest start_date, that start_date should be the first date we have data in the wys.speed_counts_agg_5kph table. This cascades into the 2 materialized views which create unique IDs for each sign installation for Open Data for mobile signs and stationary signs
e.g. for api_id 368, the start_date for the record with id=135 should be 2016-01-01 not 2019-08-16.
- [x] Update
wys.locationsso that the earlieststart_datefor eachapi_idis the first date on which we have data - [ ] Change the
wys.mobile_api_idmaterialized view to be a table with a function to insert new data monthly - [ ] Change the
wys.stationary_signsmaterialized view to be a table with a function to insert new data monthly - [ ] Add those functions to the monthly WYS dag.
- [ ] delete the old materialized views.
Noting that updates to the addresses of the following signs didn't get updated (in the mat view). api_id IN (33956, 33953).
I ended up editing the earlier address to be the new, corrected, address.
deleted the insertion functions for wys.mobile_api_id and wys.stationary_signs from wys_2022_upgrades as these mat views haven't been converted yet into tables (437a03742f207e780ca9e47cf5a58e7bc331354f)