Not clear docs on how to use jobs `autoRun`

[BorsBorsen]

Documentation Issue

(those questions were taken from dialog with copilot. unfortunately it was not able to give the answers)

how do we set an operation to automatically run every monday at 12 am with payloadcms v3 (v3.16+, since the released autoRun option) always without any user interaction? as i know it is based on "tasks", and we need to use "autoRun" property in "jobs" in "payload.config.ts", but i don't understand how to set it all together

the queue is not the slug, i was in doubts too, but found the following: https://github.com/payloadcms/payload/blob/9684d3165ee71592825536b4c5fd7ffb4f518381/packages/payload/src/index.ts#L737 https://github.com/payloadcms/payload/blob/main/packages/payload/src/queues/localAPI.ts#L72 https://github.com/payloadcms/payload/blob/main/packages/payload/src/queues/operations/runJobs/index.ts#L43 and not so obvious documentation at first, but now i understand it: https://payloadcms.com/docs/jobs-queue/queues "By specifying the queue name when you queue a new job using payload.jobs.queue() ..."

so you see, it uses queue is the thing we set with payload.jobs.queue(), and it is saving to payload-jobs mongodb collection, from where is taken on runJobs()

what i don't yet understand though, are 2 things

1. "workflows" (in contrast to simpler "tasks") also have the queue, but i am not sure if it will work out, docs are not obvious at all, so i am going to stick with the propsed payload.jobs.queue() approach
2. do we need to call payload.jobs.queue() only once in lifetime, so it will not pollute our payload-jobs with duplicating invocations? or is it safe to set it within onInit property in payload.config.ts, or what would be better to make the jobs start executing without any manual interactions?

do we need to call await payload.jobs.queue() from where? one option i see is to call it from onInit in payload.config.ts. with deleteJobOnComplete: true or without, do i need to "queue" it every time again once it is run if i want it to run forever every monday at 12 am?

---

thank you for the available functionality, i need to find out how to apply it properly

[ivkrpv]

hi! any updates/info about this? have the same question basically.

i have a task which i defined in payload.config.ts and which i want to run as a cron job on vercel (the schedule is defined in vercel.json). but what else should i do? should i add the task also to a queue? like in the docs:

const createdJob = await payload.jobs.queue({
  task: 'createPost',
})

if yes, then where should i do that, in onInit?

[g0rdonL]

I also tried to set deleteJobOnComplete to false. The cron job only ran once. Are we expected to enqueue the job again or is this a bug?

[sarwonous]

To execute the same workflow periodically, let's say every 9 PM, is it necessary to use a cron job to repeatedly enqueue it, or am I missing something?

[overdub60]

So, I'm currently fighting with the same issue. From my understanding and testing, the autorun configuration just sets up a cron schedule and processes whatever tasks are currently in the coresponding queue. We need to manually queue those tasks or workflows which I have done in the onInit() handler in the payload config. But, the problem is that I haven't found a way to re-queue the workflow or task once it finishes processing. I tried doing that in the onSuccess() handler of the task but after looking at Payload's source code it looks like the onSuccess handler is executed before the existing task is actually cleaned from the payload-jobs collection. What I am trying to prevent is to have multiple jobs of the same task in the queue at the same time, so I was checking the payload-jobs collection for an existing job and wanted to add a new one only if it didn't yet exist. But that doesn't work with the existing logic. @AlessioGr , maybe you can help us solve this mystery?

[overdub60]

I also tried to set deleteJobOnComplete to false. The cron job only ran once. Are we expected to enqueue the job again or is this a bug?

From what I understand, setting deleteJobOnComplete to false only prevents the system from cleaning up the record in the payload-jobs collection. The fields inside, e.g. totalTried and processing remain the same. Since Payload only runs a job if completedAt isn't set, if it's not processing and it didn't error, the job does not run again even though it is still in the collection – because completedAt will be set.

[iamacup]

I think the docs are really unclear about this but i believe the expected use is this:

1. Setup your queues in your autoRun

autoRun = [
      {
        cron: '0 * * * *',
        limit: 100,
        queue: 'hourly',
      },
      {
        cron: '0 0 * * *',
        limit: 100,
        queue: 'daily',
      },
      {
        cron: '* * * * *',
        limit: 100,
        queue: 'minutely',
      },
    ]

2. Queue your job somewhere (i use the init function for payload, but be aware that this function is unreliable because of nextjs and you need to make sure you can reliably run it, i modified the docker image so the run command hits the admin ui)

payload.jobs.queue({
    queue: 'hourly',
    task: 'yourTaskName',
    input: {},
  })

3. When your job is going to successfully complete, re-queue it with the same code just before it returns.

it should be noted that the point of the queues is to just run any jobs on them on the schedule they have - not to handle putting jobs onto the queue.

the deleteJobOnComplete flag should only be used if you want to look in the payload-jobs collection in the future to audit what run and when it does not provide auto re-run functonality.

[denolfe]

Let's do some improvements on these docs @AlessioGr

[Chthol1y]

it should be noted that the point of the queues is to just run any jobs on them on the schedule they have - not to handle putting jobs onto the queue.

Taking Celery's Celery Beat / Worker architecture as an example, if the worker is started with payload jobs:run --queue QUEUENAME, who is responsible for the beat? Or does such a mechanism not exist?

[christopherpickering]

It would be cool if req was passed to the onsuccess function so we could queue the next job there:

https://github.com/payloadcms/payload/blob/aef4f779b1e4487131a9aea32f1379326bab5b33/packages/payload/src/queues/operations/runJobs/runJob/getRunTaskFunction.ts#L358

[matthijs166]

Is it just me, or does the autoRun cron stop working in dev mode after HMR reload?

[AlessioGr]

Is it just me, or does the autoRun cron stop working in dev mode after HMR reload?

@matthijs166 known issue, this will be fixed once https://github.com/payloadcms/payload/pull/12863 is merged - ETA early next week.

[matthijs166]

known issue, this will be fixed once #12863 is merged - ETA early next week.

@AlessioGr very awesome!!!

Looks like you also are tackling #11611 and #12965 with this PR

[shamimevatix]

Is it just me, or does the autoRun cron stop working in dev mode after HMR reload?

same problem. It has become a nightmare. Auto run is not working at all. Why people are jumping for this immature payload cms is beyond my understanding. Rails is, was 10000 times better.

[matthijs166]

Hey @shamimevatix! The jobs system is still quite new. The HRM reload issue has been fixed in the latest Payload version (v3.49.0).

The new scheduling feature is really nice! https://payloadcms.com/docs/jobs-queue/schedules

Everyone’s open to improvements via PRs, and if Rails is a better fit for your needs, definitely go with Rails.

One concept that took me a while to grasp (without reading the docs first) is that you need to push a task or workflow onto the queue and then trigger the processing manually or using autorun.

Also, I’m still figuring out the use case in failed tasks in a workflow and retries, and choosing when to use tasks or workflows is a bit confusing too.

One thing I do miss is beforeAutorun and afterAutorun hooks. For example, I imported 1,000 items into a collection, and for each item I need to run some processing. The processing time varies per item, and I want to limit how many jobs are running at the same time. Right now, I’m working around this by using a scheduled job that checks how many jobs are currently running (or about to run), and then transfers more jobs into the active queue based on that.

That said, the foundation is very solid, and I think over time all the nice-to-haves and remaining kinks will be worked out.

If you share your config, I’d be happy to take a look and help debug what’s going wrong!

[shamimevatix]

Hi @matthijs166, sorry for my comment. I tried the latest version. But could not make the auto run work. Maybe I am missing something.

I have uploaded the sample project here https://github.com/shamimevatix/payload-cms-autorun

payload config https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/payload.config.ts#L81 `jobs: { access: { run: ({ req }: { req: PayloadRequest }): boolean => { // Allow logged in users to execute this endpoint (default) if (req.user) return true

    // If there is no logged in user, then check
    // for the Vercel Cron secret to be present as an
    // Authorization header:
    const authHeader = req.headers.get('authorization')
    return authHeader === `Bearer ${process.env.CRON_SECRET}`
  },
},
// ... other job settings
jobsCollectionOverrides: ({ defaultJobsCollection }) => {
  if (!defaultJobsCollection.admin) {
    defaultJobsCollection.admin = {}
  }

  defaultJobsCollection.admin.hidden = false
  return defaultJobsCollection
},
tasks: [
  {
    // Configure this task to automatically retry
    // up to two times
    retries: 2,

    // This is a unique identifier for the task

    slug: 'createCar',

    // These are the arguments that your Task will accept
    inputSchema: [
      {
        name: 'title',
        type: 'text',
        required: true,
      },
    ],

    // These are the properties that the function should output
    outputSchema: [
      {
        name: 'carID',
        type: 'number',
        required: true,
      },
    ],

    // This is the function that is run when the task is invoked
    handler: async ({ input, job, req }) => {
      const newCar = await req.payload.create({
        collection: 'cars',
        req,
        data: {
          title: input.title,
        },
      })
      return {
        output: {
          carID: newCar.id,
        },
      }
    },
  } as TaskConfig<'createCar'>,
],
autoRun: [
  {
    //cron: '0 * * * *', // every hour at minute 0
    cron: '*/1 * * * *', // Every minute for the queue
    limit: 10, // limit jobs to process each run
    queue: 'freeQueue', // name of the queue
  },
  // add as many cron jobs as you want
],
shouldAutoRun: async () => {
  console.log('[AutoRun] shouldAutoRun was called')
  return true
},

},`

I have never seen the log "shouldAutoRun was called"

I am trying to create a collection item through cron https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/collections/Cars/index.ts

A simple component to create the job queue item https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/components/custom/CreateJobButton.tsx

Rendering it in the post details https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/app/(frontend)/posts/%5Bslug%5D/page.tsx#L63

API route to create the job https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/app/api/create-job/route.ts

I can paste all the codes here, but it will make the message more ugly.

The good thing is, if I run

npx payload jobs:run --queue freeQueue --verbose [✓] Pulling schema from database... [16:18:33] WARN: No email adapter provided. Email will be written to console. More info at https://payloadcms.com/docs/email/overview. [16:18:33] INFO: Running 2 jobs. new: 2 retrying: 0

It works. But it doesn't process the ques automatically.

Can you please guide me. I am stuck at this point.

P.S. I am using postgres and running the project with pnpm dev. Not running in the production.

[SonNguyenne]

Can I ask something about create jobs? Do I have to create job manually in collection payload jobs, this way works for me, but I want it automatically create jobs and run it? Is there any way to do it ?

[drepkovsky]

Hi @matthijs166, sorry for my comment. I tried the latest version. But could not make the auto run work. Maybe I am missing something.

I have uploaded the sample project here https://github.com/shamimevatix/payload-cms-autorun

payload config https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/payload.config.ts#L81 `jobs: { access: { run: ({ req }: { req: PayloadRequest }): boolean => { // Allow logged in users to execute this endpoint (default) if (req.user) return true

    // If there is no logged in user, then check
    // for the Vercel Cron secret to be present as an
    // Authorization header:
    const authHeader = req.headers.get('authorization')
    return authHeader === `Bearer ${process.env.CRON_SECRET}`
  },
},
// ... other job settings
jobsCollectionOverrides: ({ defaultJobsCollection }) => {
  if (!defaultJobsCollection.admin) {
    defaultJobsCollection.admin = {}
  }

  defaultJobsCollection.admin.hidden = false
  return defaultJobsCollection
},
tasks: [
  {
    // Configure this task to automatically retry
    // up to two times
    retries: 2,

    // This is a unique identifier for the task

    slug: 'createCar',

    // These are the arguments that your Task will accept
    inputSchema: [
      {
        name: 'title',
        type: 'text',
        required: true,
      },
    ],

    // These are the properties that the function should output
    outputSchema: [
      {
        name: 'carID',
        type: 'number',
        required: true,
      },
    ],

    // This is the function that is run when the task is invoked
    handler: async ({ input, job, req }) => {
      const newCar = await req.payload.create({
        collection: 'cars',
        req,
        data: {
          title: input.title,
        },
      })
      return {
        output: {
          carID: newCar.id,
        },
      }
    },
  } as TaskConfig<'createCar'>,
],
autoRun: [
  {
    //cron: '0 * * * *', // every hour at minute 0
    cron: '*/1 * * * *', // Every minute for the queue
    limit: 10, // limit jobs to process each run
    queue: 'freeQueue', // name of the queue
  },
  // add as many cron jobs as you want
],
shouldAutoRun: async () => {
  console.log('[AutoRun] shouldAutoRun was called')
  return true
},

},`

I have never seen the log "shouldAutoRun was called"

I am trying to create a collection item through cron https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/collections/Cars/index.ts

A simple component to create the job queue item https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/components/custom/CreateJobButton.tsx

Rendering it in the post details https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/app/(frontend)/posts/%5Bslug%5D/page.tsx#L63

API route to create the job https://github.com/shamimevatix/payload-cms-autorun/blob/main/src/app/api/create-job/route.ts

I can paste all the codes here, but it will make the message more ugly.

The good thing is, if I run

npx payload jobs:run --queue freeQueue --verbose [✓] Pulling schema from database... [16:18:33] WARN: No email adapter provided. Email will be written to console. More info at https://payloadcms.com/docs/email/overview. [16:18:33] INFO: Running 2 jobs. new: 2 retrying: 0

It works. But it doesn't process the ques automatically.

Can you please guide me. I am stuck at this point.

P.S. I am using postgres and running the project with pnpm dev. Not running in the production.

Looking inside payload source code that is responsible for bootstraping the autoRun, there is this line. Notice the "options.cron" which is by default undefined when buildConfig is called

i've managed to have autoRun running by adding this snippet to instrumentation.ts

after that autoRun is working, but this behaviour isn't documented, atleast i didn't find it

[naimRahman08]

@SonNguyenne, Maybe you have to set the schdule property on the task and create a cron job to schedule job (enque job) daily and then run it.

For my application which is deployed on vercel, I have 2 endpoint for cron jobs /api/payload-jobs/handle-schedules and /api/payload-jobs/run which are already mounted by payloadcms. I have scheduled the jobs 1 hour before executing it. So no need for manual job creation.

Honestly it took me sometime to make it work as I would not have been able to do it until I read till the lastpage https://payloadcms.com/docs/jobs-queue/schedules

[slavanossar]

Just wanted to share my learnings regarding the Jobs Queue, some of it has already been mentioned in this thread but wanted to lay it out all in one place, might be useful to some.

Job A task or a workflow that has been added to a queue. They can either be created manually with payload.jobs.queue(), or automatically by the schedule property on a task/workflow.

Queue Contains queued jobs, and will only run when you trigger all queues to run with payload.jobs.run(), when you trigger a specific queue to run with payload.jobs.run({ queue: 'myQueue' }), or when you define an autoRun config that defines when that queue runs.

autoRun The autoRun config just tells Payload when it should run the specified queue. If there are no jobs in the queue, nothing will happen. Also responsible for queueing any tasks/workflows that specify a schedule with matching queue.

schedule This property is used to schedule recurring jobs. cron defines the actual schedule of the job (e.g. every day at 10AM).

So as an example, say you wanted to schedule an email to send out every day at 10AM.

// payload.config.ts

export default buildConfig({
  jobs: {
    autoRun: [
      {
        queue: 'dailyEmail',
        cron: '0 * * * *', // Every hour, every day
      },
    ],
  },
}

// jobs/workflows/sendDailyEmail.ts

const sendDailyEmail: WorkflowConfig<'sendDailyEmail'> = {
  slug: 'sendDailyEmail',
  queue: 'dailyEmail',
  schedule: [
    {
      cron: '0 10 * * *', // Daily, 10:00 AM
      queue: 'dailyEmail',
    },
  ],
  handler: async ({ job, task, req }) => {
    // ...
  }
}

Now let's step through an example timeline to see what happens (you can also read about the scheduling lifecycle here):

12:30PM We deploy our changes and Payload starts up.

1PM

• autoRun triggers the dailyEmail queue, but since no jobs are queued nothing runs.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there isn't one, the job is added to the queue with the waitUntil timestamp set to 10AM the following day.

2PM -> 9AM

• autoRun triggers the dailyEmail queue, the job for the sendDailyEmail workflow is in the queue, but its waitUntil condition isn't met so it doesn't run.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there is a scheduledRun with that slug, it skips adding a job to the queue.

10AM

• autoRun triggers the dailyEmail queue, and now the job runs since the waitUntil condition is met.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there is a scheduledRun with that slug (the one that just started running), it skips adding a job to the queue.

11AM

• autoRun triggers the dailyEmail queue, but since no jobs are queued nothing runs.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there isn't one, the job is added to the queue with the waitUntil timestamp set to 10AM the following day.

etc.

[d717an]

Just wanted to share my learnings regarding the Jobs Queue, some of it has already been mentioned in this thread but wanted to lay it out all in one place, might be useful to some.

Job A task or a workflow that has been added to a queue. They can either be created manually with payload.jobs.queue(), or automatically by the schedule property on a task/workflow.

Queue Contains queued jobs, and will only run when you trigger all queues to run with payload.jobs.run(), when you trigger a specific queue to run with payload.jobs.run({ queue: 'myQueue' }), or when you define an autoRun config that defines when that queue runs.

autoRun The autoRun config just tells Payload when it should run the specified queue. If there are no jobs in the queue, nothing will happen. Also responsible for queueing any tasks/workflows that specify a schedule with matching queue.

schedule This property is used to schedule recurring jobs. cron defines the actual schedule of the job (e.g. every day at 10AM).

So as an example, say you wanted to schedule an email to send out every day at 10AM.

// payload.config.ts

export default buildConfig({ jobs: { autoRun: [ { queue: 'dailyEmail', cron: '0 * * * *', // Every hour, every day }, ], }, } // jobs/workflows/sendDailyEmail.ts

const sendDailyEmail: WorkflowConfig<'sendDailyEmail'> = { slug: 'sendDailyEmail', queue: 'dailyEmail', schedule: [ { cron: '0 10 * * *', // Daily, 10:00 AM queue: 'dailyEmail', }, ], handler: async ({ job, task, req }) => { // ... } } Now let's step through an example timeline to see what happens (you can also read about the scheduling lifecycle here):

12:30PM We deploy our changes and Payload starts up.

1PM

• autoRun triggers the dailyEmail queue, but since no jobs are queued nothing runs.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there isn't one, the job is added to the queue with the waitUntil timestamp set to 10AM the following day.

2PM -> 9AM

• autoRun triggers the dailyEmail queue, the job for the sendDailyEmail workflow is in the queue, but its waitUntil condition isn't met so it doesn't run.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there is a scheduledRun with that slug, it skips adding a job to the queue.

10AM

• autoRun triggers the dailyEmail queue, and now the job runs since the waitUntil condition is met.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there is a scheduledRun with that slug (the one that just started running), it skips adding a job to the queue.

11AM

• autoRun triggers the dailyEmail queue, but since no jobs are queued nothing runs.
• It sees there's a workflow that has a schedule for the dailyEmail queue, so it checks the payload-job-stats global for an existing scheduledRun.
• Since there isn't one, the job is added to the queue with the waitUntil timestamp set to 10AM the following day.

etc.

This is extremely helpful. Thank you! I did not realize there was a waitUntil parameter, so I thought I had to synchronize the cron of my autoRun property to make sure it runs after my task schedule. In reality, I want it to run more frequently to capture any jobs that need to be added to the queue.

I will try increasing the frequency of my autoRun cron. Hopefully, that makes the jobs behavior a bit more predictable for us. I'm also confused as to whether or not I need to "prime" my payload app by calling something like const payload = await getPayload({ config, cron: true }) per the discussion in 13433.

[shamimevatix]

Maybe it's only me, who is agitated by their terrible documentation. There is no fully working example regarding this. Was it too hard to create an example app where a simple cron will run every minute?

[slavanossar]

Maybe it's only me, who is agitated by their terrible documentation. There is no fully working example regarding this. Was it too hard to create an example app where a simple cron will run every minute?

If you want to do that, set up an autoRun that runs every 30 seconds:

// payload.config.ts

export default buildConfig({
  jobs: {
    tasks: Object.values(tasks),
    autoRun: [
      {
        queue: 'everyMinute',
        cron: '*/30 * * * * *',
      },
    ],
  },
})

and then set up a schedule for the task/workflow that runs every minute:

const myTask: TaskConfig<'myTask'> = {
  slug: 'myTask',
  schedule: [
    {
      cron: '* * * * *',
      queue: 'everyMinute',
    },
  ],
}

This way you will run the queue (and job) when seconds tick over to hh:mm:00, and the scheduler will queue the next job when the seconds tick over to hh:mm:30.

Also I agree that the concepts of autoRun and schedule and their overlap can be a bit confusing, but the documentation actually clearly lays out how all of it works.

[shamimevatix]

Hi @slavanossar,

Thank you so much for your time. Tried the same thing. Instead of 30 seconds I applied 10 seconds. And started the project by pnpm dev, nothing in the log or in db. Here is the the file https://github.com/shamimevatix/payloadcms-cron/blob/main/src/payload.config.ts#L100

And the expression "/10 * * * * " isn't valid according to https://crontab.guru/#/10_*__*

It's not creating any new job per 30s. If I create the job manually using API endpoint, it runs the job fine.

Not sure what did I miss again.

[slavanossar]

@shamimevatix

Unix cron doesn't allow for seconds, but newer formats let you specify them and it definitely works with autoRun & schedule.

I pulled your repo and it works fine for me:

payloadcms-cron on  main is 📦 1.0.0 via ⬢ v22.19.0 took 18.1s
➜ pnpm run dev

> [email protected] dev /Users/slavanossar/Repositories/slavanossar/payloadcms-cron
> cross-env NODE_OPTIONS=--no-deprecation next dev

   ▲ Next.js 15.4.4
   - Local:        http://localhost:3000
   - Network:      http://192.168.1.2:3000
   - Environments: .env

 ✓ Starting...
 ✓ Ready in 3.5s
[00:01:26] INFO: Database "payloadcms-cron" does not exist, creating...
[00:01:26] INFO: Created database "payloadcms-cron"
[⣷] Pulling schema from database...
[✓] Pulling schema from database...
[00:01:27] WARN: No email adapter provided. Email will be written to console. More info at https://payloadcms.com/docs/email/overview.
 ✓ Compiled /admin/[[...segments]] in 7.6s (3968 modules)
Generating import map
No new imports found, skipping writing import map
GET /admin 200 in 9748ms
 ○ Compiling /api/[...slug] ...
 ✓ Compiled /api/[...slug] in 873ms (3957 modules)
 GET /api/users/me 200 in 2207ms
 GET /admin/login 200 in 233ms
 GET /admin/login 200 in 46ms
 GET /admin/create-first-user 200 in 69ms
 GET /admin/create-first-user 200 in 26ms
shouldAutoRun was called...
shouldAutoRun was called...
shouldAutoRun was called...
[00:02:00] INFO: Running 1 jobs.
    new: 1
    retrying: 0
Running job 1 to create a new post titled undefined
shouldAutoRun was called...
shouldAutoRun was called...
shouldAutoRun was called...
shouldAutoRun was called...
shouldAutoRun was called...
shouldAutoRun was called...
[00:03:00] INFO: Running 1 jobs.
    new: 1
    retrying: 0
Running job 2 to create a new post titled undefined

Note that in dev mode you need to visit /admin to trigger compilation, and this also starts up the jobs queue (and autoRun). In production it will start up when you start next.

[shamimevatix]

Thanks again for your time @slavanossar . Finally, it worked for me. But I had to update the schedule param like this

schedule: [ { cron: '*/10 * * * * *', queue: 'everyMinute', hooks: { beforeSchedule: async ({ queueable, req }) => { console.log('createPost before schedule hook called...') // Allow up to 3 simultaneous scheduled jobs and set dynamic input return { shouldSchedule: true, input: { title: Hi there..${Date.now()} }, } }, }, }, ],

Without setting shouldSchedule: true, it wasn't creating any job. If anyone is suffering like me to run a periodical job, read this one please https://payloadcms.com/docs/jobs-queue/schedules#customizing-concurrency-and-input-advanced

And here is the minimal working version: https://github.com/shamimevatix/payloadcms-cron/blob/main/src/payload.config.ts#L79

Thanks again everyone for helping me out of this misery.

[stefan-golus]

i have the same confusion.

until i got the autorun is working after admin page visit - my scheduled workflow doesn't auto-equeue jobs to the given queue. Manual ping /api/payload-jobs/handle-schedules gives me

{"message":"Cannot handle schedules because no tasks or workflows with schedules are defined."}

const endExpiredAuctionWorkflow = {
  slug: 'end-expired-auction-workflow',
  label: 'Zamykanie zakończonych aukcji',
  schedules: [
    {
      cron: '*/10 * * * * *',
      queue: 'default',
      limit: 1,
      hooks: {
        beforeSchedule: async ({ queueable, req }) => {
          const runnableOrActiveJobsForQueue = await countRunnableOrActiveJobsForQueue({
            queue: queueable.scheduleConfig.queue,
            req,
            taskSlug: queueable.taskConfig?.slug,
            workflowSlug: queueable.workflowConfig?.slug,
            onlyScheduled: true,
          })
          console.log([AutoRun] beforeSchedule was called)
          return {
            shouldSchedule: runnableOrActiveJobsForQueue < 3,
            input: { text: 'Hi there' },
          }
        },
      },
    },
  ],
  inputSchema: [],
  handler: [...]

With config

jobs: {
    jobsCollectionOverrides: ({ defaultJobsCollection }) => {
      if (!defaultJobsCollection.admin) {
        defaultJobsCollection.admin = {}
      }
      defaultJobsCollection.admin.hidden = false
      return defaultJobsCollection
    },
    tasks: [
      sendVehicleAcceptanceEmailTask,
      createAuctionTask,
      generateVehicleDescriptionTask,
      draftListingFollowupNotificationTask,
      endExpiredVehicleAndAuctionTask,
      sendEmailToEndedAuctionSellerTask,
      sendEmailToEndedAuctionWinnerTask,
      sendEmailToEndedAuctionBiddersTask,
    ],
    workflows: [vehicleAcceptanceWorkflow, endExpiredAuctionWorkflow],
    autoRun: [
      {
        cron: '*/5 * * * * *', 
        limit: 10,
        queue: 'default',
      },
    ],
    shouldAutoRun: async (_payload: Payload) => {
      // Tell Payload if it should run jobs or not. This function is optional and will return true by default.
      // This function will be invoked each time Payload goes to pick up and run jobs.
      // If this function ever returns false, the cron schedule will be stopped.
      console.log('[AutoRun] shouldAutoRun was called')
      return true
    },
  }

[slavanossar]

@stefan-golus The property on a task or workflow is schedule not schedules

[amitabhdhiwal]

Here's what I've learnt. First, you need to visit '/admin' to trigger recompiling, and that your dev mode mileage will vary.

Now, A Task is something you want to do later. A Workflow is a set tasks.

A Job is first created, and then run.

Manual Jobs A Job is created when a task or a workflow is queued by calling the payload.jobs.queue method.

A job is run by calling the payload.jobs.run, or payload.jobs.runByID methods.

Automated Jobs A Job is automatically created when a workflow config has queue:, and a schedule: [] defined. schedule key has these properties: queue, cron. The job will be automatically created as per this cron rule in the queue. So in a workflow config, a cron of 0 * * * * and queue named i_will_do_this, will create a job every hour in the queue i_will_do_this.

A Job is automatically run when the payload config has jobs.autoRun: [] defined with queue, cron, and limit. limit number of jobs from the queue will be automatically run every cron rule. So in the payload config, a autoRun cron of 0 * * * * for the queue i_will_do_this will automatically run the jobs in that queue every hour. Maximum jobs run per hour will be the value of limit for that queue.

The cron string accepts the seconds part, which is optional. So * * * * * and * * * * * * both are valid.

Implementation If you want to manually create jobs, and manually run them:

1. Create the job with payload.jobs.queue, which will return a job object which will have an id.
2. Run the job by calling payload.jobs.runByID.
3. Visit '/admin' to trigger recompiling

If you want to manually create jobs, and automatically run them at different times:

1. Create the job with payload.jobs.queue with waitUntil date, which will return a job object which will have an id.
2. Run the job by calling payload.jobs.runByID, or payload.jobs.run.
3. Visit '/admin' to trigger recompiling

If you want to manually create jobs, but automatically run them at fixed interval:

1. In your payload config, define jobs.autoRun with a "queue" name, and "cron"
2. When calling payload.jobs.queue pass the queue name as parameter
3. Visit '/admin' to trigger recompiling

If you want to automatically create jobs, but manually run them:

1. In your workflow config, define the "schedule" property with cron and queue name
2. Call payload.jobs.run with the queue name later through the cli or the http or within code somehow
3. Visit '/admin' to trigger recompiling

If you want to automatically create jobs, and automatically run them at fixed interval:

1. In your payload config, define jobs.autoRun with a "queue" name, and "cron".
2. In your workflow config, define the "schedule" property with cron and queue name
3. Visit '/admin' to trigger recompiling

Hope this helps. The documentation is like an RPG. 🤷‍♂️