redash icon indicating copy to clipboard operation
redash copied to clipboard

Published 20 hours ago • verified publisher icon getredash

Weekly Scheduled Jobs not Running

Open ganeshapp opened this issue 4 years ago • 7 comments

Issue Summary

Weekly Scheduled Queries not running for Postgres, Presto and Redshift.

Steps to Reproduce

  1. Create new query in Redash V9. Beta
  2. Save, publish, execute it.
  3. Schedule it for 1 Week job, repeating every Sunday or Monday (any day) etc. and select time 8:00 AM (any time)
  4. Is not queued on Sunday 8:00 AM (day and time is just example).

Any other info e.g. Why do you consider this to be a bug? What did you expect to happen instead? Same query scheduled to run every 30 minutes or every 1 day at 8:00 AM gets queued and executed fine. Same Query scheduled to run 1W on Sunday 8:00 AM runs fine on Redash V8.

Technical details:

ganeshapp avatar Jul 27 '20 17:07 ganeshapp

We're seeing the same behavior too. Happy to submit a PR if anyone has a clue where this goes wrong?

itssimon avatar Jul 28 '20 09:07 itssimon

I have the same problem.how to solve it?

yyzhou0811 avatar Jul 29 '20 02:07 yyzhou0811

can not wait for the new version

xuanyuanaosheng avatar Jul 30 '20 05:07 xuanyuanaosheng

@ganeshapp Is it related to the JOB EXPIRY TIME (by default, it is 3600 * 12 seconds) ? https://github.com/getredash/redash/blob/90024ebc92ebd952d7ec8c085eb70132642564da/redash/tasks/queries/execution.py#L113

tim5go avatar Sep 16 '20 15:09 tim5go

@tim5go If that is the case 3600 * 12 seconds = 12 hours, then even the daily jobs should fail right? But the daily jobs run. Only the weekly jobs don't get scheduled.

ganeshapp avatar Sep 16 '20 15:09 ganeshapp

JOB_EXPIRY_TIME doesn't cause this. Because the job isn't enqueued until the query actually needs to run. This is what the scheduler service does. It runs twice each minute and checks which scheduled queries need to be enqueued now.

So what causes it? I don't know. But I've just self-assigned this and will investigate.

susodapop avatar Dec 06 '21 15:12 susodapop

Could it be something like this: https://github.com/solute/redash/commit/1a3e13084e7b26c9c0b0e49730b115b5c66332a2

I stumble upon this while fumbling around with parameterized queries (before I read in the docs scheduled parameterized queries are not supported right now, even though you can activate a schedule, it just does not work properly if you do).

So while hunting down my perceived "bug" I found something that seemed wrong to me and "fixed" it with the above commit. If the analysis in the commit message is stupid, I'd also be happy to know why the original code is fine, because I am obviously not getting what is going on then. :-(

mbra avatar Jan 26 '22 11:01 mbra