etl
etl copied to clipboard
Complete v2 data pipeline migration
Steps taken from: https://docs.google.com/document/d/1seI56IGAZzfIhmkZH_Pp67fU11kynyO6mwf7gU3HeiM/edit#heading=h.386cbm2zij4h
Prepare pipeline
- [x] Revert switch v2 parser from HEAD
- [x] Merge paris1 hopannotation1 changes in v1 pipeline
- [x] Deploy etl w/ hopannotation1 changes to production
- [x] Disable v1 pipeline deployments (v1 will run until we stop it)
Add step for "Begin notifying users"
- [x] Publish blog post - https://github.com/m-lab/website/pull/677
- https://www.measurementlab.net/blog/v2-data-pipeline-migration
- [x] Send email to discuss@ announcing changes
One at a time:
- [x] Remove references to undecorated legacy v1 views in etl-schema
- [x] Remove references to undecorated legacy v1 views in prometheus-support
- [x] Verify no other references are in use to the undecorated legacy v1 views
- [x] Remove obsolete library views
- [x] Remove obsolete v1 views under ndt dataset
- [x] Add legacy view to replace default views for sidestream, and utilization datatypes
- [x] https://github.com/m-lab/etl/issues/1046
- [x] Update traceroute
- [x] #1061
- [x] #1066
- [x] #1074
- [ ] https://github.com/m-lab/etl/issues/1084