probe-scraper icon indicating copy to clipboard operation
probe-scraper copied to clipboard

Refactor this Code Base

Open fbertsch opened this issue 5 years ago • 3 comments

This code base is largely about:

  • Scraping information about revisions
  • Reading probe information from those revisions
  • Combining probe information
  • Writing it back out

Given that we recently moved to Python 3, this code base could use a serious uplift by utilizing some of the nice features available. For example, the probe type could be a Dataclass that knows how to compare itself to others, and knows how to serialize itself into the final JSON output.

A revision class could compare itself to other revisions, based on push-date or version. This can be used to decide first and last revisions/versions/dates for probes.

Together, these would simplify transform_probes.py.

All logic should be moved out of runner.py and integrated into appropriate places, and that should exist just to provide the scraper CLI.

fbertsch avatar Feb 27 '19 18:02 fbertsch