register icon indicating copy to clipboard operation
register copied to clipboard

PSC-STM-B7: Switch over to live transformation

Open tiredpixel opened this issue 1 year ago • 0 comments

  • Perform live transformation on all existing S3 files produced by the Kinesis Firehose attached to the ingester
  • Start the live transformer (which will default to the start of the stream, but will skip any records which have already been published).

Following this, it should be unnecessary to keep performing bulk imports, since the live transformer should immediately transform latest data.

Estimate: 4 hours

tiredpixel avatar Feb 28 '24 17:02 tiredpixel