mimir
mimir copied to clipboard
Make Value Repair its own operator
"Value Repair" is the "repair" part of the Missing Value lens. It would be helpful to make this its own operator for two reasons:
- Missing Value Repair is something that can happen in multiple places (missing value lens, type inference lens)
- Missing Value Repair can be implemented in different ways, each involving a very different query structure: Imputation can be handled by projection, Remove bad data is a selection, Interpolation is a windowed aggregate,
Proposed structure:
ApplyValueRepair(
model: String, // Referencing ModelManager
input: Operator,
target: Var,
constraint: Constraint
)
Proposed semantics: Return a table based on [input] that (1) has an identical schema, (2) is "minimally different", and (3) for which [constraint] applied to [target] is satisfied for every row. Permitted edits include:
- Delete rows
- Modify [target]
InlineVGTerms (should be renamed now) should use [model] to select a repair to replace ApplyValueRepair with.