Flowminder
Split AutoFlow into two components?
The AutoFlow available dates sensor could in principle be modified to trigger things other than notebook-based Prefect workflows, and the workflows it triggers wouldn't necessarily need to run in the same container as the sensor.
We could split AutoFlow into two components: one that runs the dates sensor, which sends messages to a series of user-requested locations when new data is available; and another that listens for messages from the first component and runs a notebook-based workflow when triggered. This could allow more flexibility for the available dates sensor to trigger things other than notebook-based workflows, such as triggering a dashboard backend to load new data.
On the notebook-based workflow side, prefect PIN 8 (event-driven flows via listeners) would make this simpler to implement.
I kind of like that idea, although I wonder if once you're at that point you might as well go a step further an run an event service which sends a 'day x available now' message via a message broker and trigger it from flowetl.
