spyglass
spyglass copied to clipboard
Peripheral tables cause issues: deletes, field name uniqueness, etc.
By peripheral tables, I refer to tables like IntervalList
and AnalysisNwbfile
. These tables ...
- Have many foreign key references throughout various pipelines
- Centralize a data type (e.g., interval, file path)
Pros:
- Centralize processes that manage their type (e.g.,
cleanup
, interval list operations,fetch_nwb
) - Facilitate operations that compare instances, though I'm not aware of current cases.
Cons:
- Not captured by cascading deletes, requiring the use of operations like
cleanup
(see #948) - Result in redundancy (see #778) requiring workarounds like
cautious_insert
- Result in keys managing multiple values for these data types (see #961), requiring renaming
- Cause issues tracking an entry through a pipeline, providing false parent paths for
RestrGraph
An alternative design would replace foreign key references with fields and centralize enforcement of the data type via a utility.
Converting existing pipelines is impractical.