etl icon indicating copy to clipboard operation
etl copied to clipboard

Complete v2 data pipeline migration

Open stephen-soltesz opened this issue 3 years ago • 0 comments

Steps taken from: https://docs.google.com/document/d/1seI56IGAZzfIhmkZH_Pp67fU11kynyO6mwf7gU3HeiM/edit#heading=h.386cbm2zij4h

Prepare pipeline

  • [x] Revert switch v2 parser from HEAD
  • [x] Merge paris1 hopannotation1 changes in v1 pipeline
  • [x] Deploy etl w/ hopannotation1 changes to production
  • [x] Disable v1 pipeline deployments (v1 will run until we stop it)

Add step for "Begin notifying users"

  • [x] Publish blog post - https://github.com/m-lab/website/pull/677
    • https://www.measurementlab.net/blog/v2-data-pipeline-migration
  • [x] Send email to discuss@ announcing changes

One at a time:

  • [x] Remove references to undecorated legacy v1 views in etl-schema
  • [x] Remove references to undecorated legacy v1 views in prometheus-support
  • [x] Verify no other references are in use to the undecorated legacy v1 views
  • [x] Remove obsolete library views
  • [x] Remove obsolete v1 views under ndt dataset
  • [x] Add legacy view to replace default views for sidestream, and utilization datatypes
  • [x] https://github.com/m-lab/etl/issues/1046
  • [x] Update traceroute
  • [x] #1061
  • [x] #1066
  • [x] #1074
  • [ ] https://github.com/m-lab/etl/issues/1084

stephen-soltesz avatar Feb 04 '22 15:02 stephen-soltesz