spyglass
spyglass copied to clipboard
CurationV1 does not track a metrics source.
Describe the bug
- The table
CurationV1
sources metrics manually passed by the user ininsert_curation
to make the table entries. - As far as I can tell, there is no way to go back and trace where those metrics came from
- makes it hard for reproducibility
- adds risk of data integrity due to user error at insert
- makes it hard to select a Curation entry based on upstream features in the future
Solutions Brainstorm
- The v0 pipeline had a similar need to not make a formal dependency on metric source in
Curation
- data integrity due to user error partially solved by making the population of
Curation
done inAutomaticCuration.populate
. - Could potentially add similar functionality in the population of
MetricCuration
- data integrity due to user error partially solved by making the population of