[Core] Airflow scheduler as CRON

[nathan-vm]

Today, we utilize the start_date + schedule interval defined by Airflow. However, this can lead to unexpected behavior, such as:

start_date=20:36 + @hourly = next run at 22:00 because at 21:00 it does not elapse one hour after the start date, resulting in the next schedule being set for the following hour.

Another issue is a UX-related problem: with the default Airflow schedule interval, it is not possible to use fractions of time (like run in 21:36). Below is the reference table:

For the UX issue, we need to discuss if it is an expected behavior or not.