bdit_data-sources
bdit_data-sources copied to clipboard
Miovision Aggregation Improvements (ATR view, intersection_movements AFTER INSERT trigger, unacceptable_gaps)
What this pull request accomplishes:
- Remove
miovision_api.volumes_15min
(table) in place of view (volumes_15min_atr_filtered
). Reduces duplicate storage (~75GB) and complexity in aggregation. - Add an
AFTER INSERT
trigger tointersection_movements
to help aggregate and pad new movements into volumes_15min_mvt.- volumes/miovision/sql/function/function-add_intersection_movement_padding_values.sql: based on existing inserts into volumes_15min_mvt, except using
new_rows
(transition relation) instead ofintersection_movements
and without date limits.
- volumes/miovision/sql/function/function-add_intersection_movement_padding_values.sql: based on existing inserts into volumes_15min_mvt, except using
- Move
unacceptable_gap
removal from table inserts to filtered view: less complexity in adding and removing unacceptable_gaps.
Summary of files changed: Many files have minor changes including renamed references, removal of ATR aggregation, big fluff (intersection_movements, volumes_15min_tmc).
Files with major changes:
- volumes/miovision/sql/function/function-add_intersection_movement_padding_values.sql: trigger function to add values to 15min aggregate table based on additions to intersection_movements. Based closely on existing aggregation but only for new intersection_movements.
-
volumes/miovision/sql/table/create-table-movement_map_new.sql: a wide version of
movement_map
with entry/exit leg columns and directions more clearly labelled. Helps with ATR view joins. - volumes/miovision/sql/views/create-view-volumes_15min_atr_filtered.sql: ATR view which replaces table.
- volumes/miovision/sql/views/create-view-volumes_15min_mvt_filtered.sql: add anti-join of unacceptable_gaps.
Issue(s) this solves:
- Closes #868, #879: filters
unacceptable_gaps
via view - Progress towards closing issues around
intersection_movements
: #406, #855
What, in particular, needs to reviewed:
- [ ] Another option considered is adding movement map columns
entry_dir, exit_leg, exit_dir
to TMC table so ATR view wouldn't need to join to movement_map. Downside: more confusing TMC table, does not match current TMC table. - [ ] Ok to anti join instead of set unacceptable_gaps to null in filtered view?
- [ ] New naming: match existing to reduce dependency changes or make contents more explicit?
Old | New | |
---|---|---|
TMC | volumes_15min_mvt | volumes_15min_mvt_unfiltered |
volumes_15min_mvt_filtered | volumes_15min_mvt_filtered | |
ATR | volumes_15min | |
volumes_15min_filtered | volumes_15min_atr_filtered |
What needs to be done by a sysadmin after this PR is merged
E.g.: these tables need to be migrated/created in the production schema.