superduper icon indicating copy to clipboard operation
superduper copied to clipboard

[QUEUES] Allow jobs created on `Component` creation to be passed onto downstream components

Open blythed opened this issue 7 months ago • 8 comments

  1. Compute features
  2. Train PCA on finished computed features
  3. Compute dimension reduced features

class DimReduceModel(Model):
    trainer: ...
    upstream: ...

listener1 = Listener(features_model, ...)
dim_reduce_model = DimReduceModel(trainer=PCATrainer(), upstream=[listener1])
listener2 = Listener(dim_reduce_model, ...)

Explanation

  • The listener1 creates features
  • dim_reduce_model trains on those features
  • listener2 creates features after the training is complete
Screenshot 2024-07-18 at 9 21 18 PM

blythed avatar Jul 18 '24 11:07 blythed