telemetry-analysis-service icon indicating copy to clipboard operation
telemetry-analysis-service copied to clipboard

Job "modules-with-missing-symbols" has due date in the past

Open jezdez opened this issue 7 years ago • 4 comments

from a comment in #553:

This is still happening on my "modules-with-missing-symbols" job.

"It's due next at 2017-12-04 14:25 UTC.", which is yesterday. It was last run on December 1. It is scheduled weekly, so the next date should be December 8.

jezdez avatar Dec 05 '17 18:12 jezdez

/cc @marco-c

jezdez avatar Dec 05 '17 18:12 jezdez

Thanks for the report @marco-c, this seems indeed odd at first, but one thing that isn't obvious to you when looking at the due date is that we had a outage last week that prevented all scheduled jobs to run.

To rectify the situation, I've had to manually run the failed job after fixing the issue causing the outage, including your weekly job.

While doing so it seems as if the next due date was not correctly set and I suspect due to a missing or throttled status update request from AWS that prevented ATMO to save the last run date. In effect the next due date wasn't able to be calculated and ATMO fell back to using the start date as the indicator to calculate it. The displayed due date was wrong in that case.

I've fixed this now by resetting the due date by saving the job once via the web ui, which triggers the due date update mechanism. Note: Since it found a due date in the past it also started the job. Once this run is over the job should be back to the regular schedule.

jezdez avatar Dec 05 '17 18:12 jezdez

I'm leaving this ticket open to follow up once the current run is done.

jezdez avatar Dec 05 '17 18:12 jezdez

@jezdez it's still happening. My job "modules-with-missing-symbols" is "due next at 2017-12-12 19:43 UTC.".

marco-c avatar Dec 19 '17 17:12 marco-c