spyglass icon indicating copy to clipboard operation
spyglass copied to clipboard

Peripheral tables cause issues: deletes, field name uniqueness, etc.

Open CBroz1 opened this issue 9 months ago • 0 comments

By peripheral tables, I refer to tables like IntervalList and AnalysisNwbfile. These tables ...

  • Have many foreign key references throughout various pipelines
  • Centralize a data type (e.g., interval, file path)

Pros:

  1. Centralize processes that manage their type (e.g., cleanup, interval list operations, fetch_nwb)
  2. Facilitate operations that compare instances, though I'm not aware of current cases.

Cons:

  1. Not captured by cascading deletes, requiring the use of operations like cleanup (see #948)
  2. Result in redundancy (see #778) requiring workarounds like cautious_insert
  3. Result in keys managing multiple values for these data types (see #961), requiring renaming
  4. Cause issues tracking an entry through a pipeline, providing false parent paths for RestrGraph

An alternative design would replace foreign key references with fields and centralize enforcement of the data type via a utility.

Converting existing pipelines is impractical.

CBroz1 avatar May 08 '24 19:05 CBroz1