raystack
Job sensor is aiming for inaccurate window of upstream
Describe the bug A daily job that depends on another daily job, does not have an accurate sensor aiming for the same data window. For example, given this case:
Job A
- Window size: 24h
- Truncate to: d
- Offset: 24h
- Schedule: every
22.00
Job B
- Window size: 24h
- Truncate to: d
- Offset 0
- Schedule: every
23.00 - Have dependency to
job A
Job B will have a sensor that is pointing to job A, with the same window. However, let's say on 2nd October run, job B sensor is checking on job A 1st October run, instead of 2nd. The impact is if Job A on 2nd October takes longer to finish (for example 23.30), job B will not wait for it to be finished as the 1st October run has been finished.
To Reproduce Create 2 jobs with details as mentioned in the bug description.
Expected behavior Sensors should be created accurately. Daily jobs depend on another daily job should wait for an equal data window.
