mozilla-pipeline-schemas icon indicating copy to clipboard operation
mozilla-pipeline-schemas copied to clipboard

Add check for BQ compatible schema-changes to PR review checklist

Open fbertsch opened this issue 6 years ago • 4 comments
trafficstars

This comes from the GCP schema handling doc.

We will need to initially start minimizing schema-incompatible changes for BQ. This only means that we can't change field types and can't remove fields. The former has historically (AFAIK) not been a common change, and the latter can be easily avoided by simply keeping the fields.

fbertsch avatar Apr 22 '19 15:04 fbertsch

Note on implementation: this test will likely need to pull the pre-PR state of any changed schemas via some git commands, then do some amount of parsing of the two versions of the schema to detect the conditions we want to avoid.

jklukas avatar Apr 23 '19 14:04 jklukas

Those are good points @jklukas! I meant for this to just be added to the PR review checklist. Mark and I were discussing a longer-term CI test for schema changes, with a short-term improvement being akin to the one you mentioned, should we move that to a separate issue?

fbertsch avatar Apr 23 '19 15:04 fbertsch

should we move that to a separate issue

Indeed. I interpreted this as being about CI tests. Apologies for missing the scope. I agree that adding to a checklist is a good first step that can be done immediately.

jklukas avatar Apr 23 '19 15:04 jklukas

Okay great. Updated the title of this issue to make it more clear, and let's move CI discussion to #317.

fbertsch avatar Apr 23 '19 15:04 fbertsch