COOL icon indicating copy to clipboard operation
COOL copied to clipboard

Issues left by cohort processor refactoring

Open Zrealshadow opened this issue 2 years ago • 1 comments

Create an issue and submit a corresponding PR for each one.

  1. need to add the logic to check the meta chunk to speed up the query processing.
  2. remove aggregation logic in ValueSelection.
    • ValueSelection only handles filtering (can be merged with EventSelection)
    • RetUnit will encapsulate the logic of updating statistics (discussed below)
  3. Factory class to handle FieldRS creation (better encapsulation)
  4. rework the ProjectedTuple
    • existing impl aims to mimic the old logic which introduces additional indirections
    • keep it simple, let its producer decides on which indexed value to retrieve from it.
    • make it immutable. avoid loadattr that mutates internal data.
    • separate handling for special field: user id and action time
  5. MetaFieldRS to create value converter/translator (extensibility and polymorphism)
    • currently we only have two types, we can expect having more field types.
    • The MetaFieldRS translates the token (gid for string now) stored in data chunk to actual values. (no-op for range)
  6. Augment RetUnit
    • perhaps we should rename this variable
    • it will contain additional variables for max, min, etc. (now only counts)
    • A solution is to have a list of variable and keep a list functions (aggregators) that take in new value and update its corresponding variable.
  7. add documentations: DataHashFieldRS Need to add descriptions to the assumptions: all input vector ever used there have efficient implementation of getting by index (Zint, BitVector, ZintBitInput,)

Zrealshadow avatar Aug 08 '22 06:08 Zrealshadow