how to execute metorikku in a loop from a single spark-submit?

[tooptoop4]

use-case: near-real time sync from MSSQL db to s3. I know I can do a while loop calling spark-submit each time but this will be slow as JVM needs to startup each time. Is there a way to make metorikku keep on looping through the same input/metric.yaml within a single spark-submit?

note: i know should have kafka..etc but not an option at this stage

[lyogev]

We don't have such a feature, but it's a cool idea to create fake micro batches in metorikku to stream on non streaming sources