probe-scraper icon indicating copy to clipboard operation
probe-scraper copied to clipboard

Change existing git repo parsing to Glean parsing

Open fbertsch opened this issue 6 years ago • 0 comments

We currently have a setup to include the types of files we expect for Desktop: Histograms.json, Scalars.yaml, etc., but for Github repos. The Glean work is replacing that effort, so we need to update the code to parse metrics.yaml files instead.

This work will encompass:

  • Parsing and writing out information on Glean probes to the probe-dictionary
  • Parsing and writing out information on Glean pings to the probe-dictionary

The former will be used for schema creation. The latter will be used for validation and table names in the pipeline.

We can use the Glean Parser library to parse the metrics.yaml files and write out results. The rationale for letting the Glean Parser serialize is then the scraper doesn't need to have intimate knowledge of which fields are required and which may be updated; instead we can just add and remove fields from the parser.

fbertsch avatar Feb 13 '19 22:02 fbertsch