dependabot-core icon indicating copy to clipboard operation
dependabot-core copied to clipboard

Support bi-weekly intervals in cron interval expressions

Open mikelkew opened this issue 7 months ago • 26 comments

Is there an existing issue for this?

  • [x] I have searched the existing issues

Package ecosystem

All

Package manager version

No response

Language version

No response

Manifest location and content before the Dependabot update

No response

dependabot.yml content

version: 2
enable-beta-ecosystems: true
updates:
  - package-ecosystem: "bundler"
    directory: "/"
    schedule:
      interval: "cron"
      cronjob: "0 8 * * 1/2"
      timezone: "Pacific/Auckland"

Updated dependency

No response

What you expected to see, versus what you actually saw

We were very happy in our team to see the new support added for the cron scheduled interval type (as introduced in https://github.com/dependabot/dependabot-core/issues/6339). We have recurring tasks to address any dependency updates on a biweekly cadence to align with our Sprints, and we were hopeful that the new cron intervals would allow us to schedule dependabot to run its checks biweekly, e.g. every 2nd Monday.

We attempted to achieve this using step values on the day-of-week field in the cron expression, e.g. "0 8 * * 1/2"

Given this expression, we would expect it to run every 2nd Monday at 8am.

However, as reported in https://github.com/dependabot/dependabot-core/issues/6339#issuecomment-2837647979, when testing it, it actually ran weekly, i.e. every Monday at 8am.

I'm aware that support for step values on the day-of-week field is not fully supported by all cron engines, however, I suspect that it would be a common requirement to schedule it to run biweekly, particularly for teams running a 2 week sprint cadence. If there's any way that this could be supported it would be very much appreciated! 🙏🏼

cc @sachin-sandhu @abdulapopoola

Native package manager behavior

No response

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

mikelkew avatar May 12 '25 15:05 mikelkew

@sachin-sandhu can you take a look please?

abdulapopoola avatar May 12 '25 15:05 abdulapopoola

Hi @mikelkew ,

Just to clarify this , are you looking to run updates every other monday (after 15 days)?

You can use 0 8 * * mon%2

It should run on following dates at 8 AM

2025-05-26 08:00:00 
2025-06-09 08:00:00 
2025-06-23 08:00:00 
2025-07-07 08:00:00 
2025-07-21 08:00:00 
2025-08-04 08:00:00

sachin-sandhu avatar May 13 '25 04:05 sachin-sandhu

Thanks @sachin-sandhu, I've updated our configuration to test it out - I'll report back here once I've validated it and close if resolved.

This modulo extension syntax is fairly non-standard. Out of curiosity (and to better understand what syntax is supported), what are you using for the cron scheduling? fugit perhaps?

mikelkew avatar May 15 '25 07:05 mikelkew

Yes, this is currently implemented using the fugit library.

jeffwidman avatar May 16 '25 05:05 jeffwidman

I've updated our configuration to test it out - I'll report back here once I've validated it and close if resolved.

@sachin-sandhu surprisingly, despite us using the following schedule config, Dependabot ran at 8am this morning (Sun, 18 May 2025 20:21:03 GMT according the the log).

schedule:
  interval: "cron"
  cronjob: "0 8 * * mon%2"
  timezone: "Pacific/Auckland"

Testing this using fugit locally, this should only run on 2025-05-26 08:00:00, not 2025-05-19 08:00:00.

Can you think of any reason why it still ran? Does the schedule get updated as soon as a PR is merged that changes the config, or is there a delay for config to be applied? Does the use of a non-UTC timezone potentially impact it?

mikelkew avatar May 19 '25 02:05 mikelkew

Hi @mikelkew , We are using fugit library for cron. what was your setup prior to cron. was it weekly monday 8.

sachin-sandhu avatar May 20 '25 13:05 sachin-sandhu

@sachin-sandhu Correct, this is what we had previously - it appears to still be running on this schedule:

schedule:
  interval: "weekly"
  day: "monday"
  time: "08:00"
  timezone: "Pacific/Auckland"

mikelkew avatar May 20 '25 23:05 mikelkew

@mikelkew ,

thanks for confirming, looks like somehow your .yml contents are not synced. Can you check if your yml CI has passed.

Image

sachin-sandhu avatar May 21 '25 02:05 sachin-sandhu

@sachin-sandhu Thanks, but we don't have a YML CI action - we're using the default GitHub Dependabot functionality, not a self-hosted version of Dependabot. Shouldn't GitHub pickup the changes to the Dependabot config and adjust the schedule automatically?

mikelkew avatar May 21 '25 02:05 mikelkew

@mikelkew , you are correct, the yml updates are synced to github automatically along with validation, You can see the validation results once you browse the file in github. we found that syncing is not working for few repos with similar issue. are you facing this issue in public repo, we can attempt to resync it and see if that resolves the issue.

sachin-sandhu avatar May 23 '25 15:05 sachin-sandhu

Thanks @sachin-sandhu, checking the actions dropdown after I've browsed to ../.github/dependabot.yml, I can't see anything there related to YML updates.. All just our own Buildkite CI actions.

Unfortunately this isn't for a public repo. Are there any other steps you may recommend to attempt to get it to resync? Could it be worth removing the dependabot.yml file completely and then adding it back?

mikelkew avatar May 25 '25 23:05 mikelkew

@mikelkew, yes, we can try removing the yml file and add it back. lets see if this fixes it.

We are looking at other resync options meanwhile.

sachin-sandhu avatar May 26 '25 12:05 sachin-sandhu

@mikelkew , adding the yml file back should run dependabot for you once. can you please attach the logs (remove any unnecessary info if needed) , thanks !

sachin-sandhu avatar May 28 '25 15:05 sachin-sandhu

Update:

@mikelkew , We have found a related fix , we are working to release it, lets wait and see if this fixes the issue.

sachin-sandhu avatar May 29 '25 03:05 sachin-sandhu

@mikelkew , we have fixed the issue, we are currently manually syncing the affected repos. it may take some time to reflect, meanwhile you can make any change in your yml file to resync . it should update the schedule type to cron.

sachin-sandhu avatar May 29 '25 22:05 sachin-sandhu

@sachin-sandhu thanks for the update. For some extra context, we tried removing the config file, and then re-adding it. However, it at that point it didn't even seem to recognise that any config file existed.

We then restored our config file back to the old interval: "weekly" setting, and the config was applied successfully. It seems like it was failing to sync purely because of the cron setting.

Thanks for investigating. Hopefully that's resolved the issue - I'll try updating our config file to use the cron syntax again and see how we go! 🤞🏼

mikelkew avatar May 29 '25 23:05 mikelkew

@sachin-sandhu unfortunately it appears that the new cron interval is still not being applied. For context, we:

  • Removed the dependabot.yml config, which completely disabled Dependabot checks
  • Restored the dependabot.yml config using our prior interval: "weekly" setting. This appeared to be detected, and the Dependabot checks were re-enabled
  • Updated the dependabot.yml config to use the following cron syntax for both bundler and npm package ecosystems.
    schedule:
      interval: "cron"
      cronjob: "0 8 * * mon%2"
      timezone: "Pacific/Auckland"
    
  • Despite that, Dependabot still ran at 2025-06-02T08:19:49+12:00 yesterday. Testing the cron expression "0 8 * * mon%2" using fugit locally (see below), it should have run next week, not this week, leading me to conclude that the settings weren't applied. Is there any chance that the runner is failing to apply this config, perhaps due to a validation issue or similar?
    irb(main):002> c = Fugit.parse("0 8 * * mon%2")
    => #<Fugit::Cron:0x000000011c1d19c0 @cron_s=nil, @day_and=nil, @hours=[8], @minutes=[0], @monthdays=nil, @months=nil, @original="0 8 * * mon%2", @seconds=[0], @timezone=nil, @weekdays=[[1, [2, 0]]], @zone=nil>
    irb(main):003> c.match?('2025-06-02 08:00')
    => false
    irb(main):004> c.match?('2025-06-09 08:00')
    => true
    

mikelkew avatar Jun 03 '25 03:06 mikelkew

@mikelkew ,

I'll check this asap. Can you please mention the job id from last run logs. It should start with <job_xxxxxxxxx>

sachin-sandhu avatar Jun 03 '25 12:06 sachin-sandhu

Thanks for looking into this @sachin-sandhu, the last scheduled job appears to have been job_1025455521, although the most recent job (triggered by a rebase) was job_1027067118.

mikelkew avatar Jun 03 '25 12:06 mikelkew

@mikelkew , we have found the related issue, we are thinking of possible solution to fix this problem. I'll update once we fix this issue, thanks !

sachin-sandhu avatar Jun 04 '25 04:06 sachin-sandhu

Also this cron is broken

schedule:
      interval: cron
      cronjob: 0 14 1-7 * MON

#or

schedule:
      interval: cron
      cronjob: 0 14 1-7 * 1

From crontrab.guru

Image

For humans : It aims to triggers on the first monday of each month.

And it triggers every day from 1 -> 7 looks like the last constraint is ignored.

Waxo avatar Jun 07 '25 17:06 Waxo

@Waxo it might be worth trying this expression instead: 0 14 * * mon#1

(as per the fugit docs)

mikelkew avatar Jun 08 '25 23:06 mikelkew

we have found the related issue, we are thinking of possible solution to fix this problem

Thanks @sachin-sandhu! I've restored our original weekly interval configuration for the time being. Please let me know once you've got a fix in place so that we can try again 🙏🏼

mikelkew avatar Jun 08 '25 23:06 mikelkew

@Waxo it might be worth trying this expression instead: 0 14 * * mon#1

(as per the fugit docs)

Seems to fix my cron, and the main problem was the missing quotes in my expression.

Thanks again !

Waxo avatar Jun 16 '25 12:06 Waxo

we have found the related issue, we are thinking of possible solution to fix this problem

Thanks @sachin-sandhu! I've restored our original weekly interval configuration for the time being. Please let me know once you've got a fix in place so that we can try again 🙏🏼

Hi @mikelkew ,

We are occupied right now with scheduled activities, I have plan to fix the issue next week.

sachin-sandhu avatar Jun 16 '25 21:06 sachin-sandhu

Thanks for the update @sachin-sandhu 🙏

mikelkew avatar Jun 16 '25 21:06 mikelkew

@mikelkew , we have fixed the issue, i tested on a test repo and fix is good. Can you check if it works.

thanks

sachin-sandhu avatar Jun 30 '25 15:06 sachin-sandhu

@mikelkew , we are closing this issue. If you see the issue happening again, you can raise/reopen this and tag me in comment.

Thanks !

sachin-sandhu avatar Jul 09 '25 21:07 sachin-sandhu

Thanks @sachin-sandhu, after giving it a couple of weeks for testing, it appears to be working well so far. Thank you! 🙏🏼

mikelkew avatar Jul 14 '25 16:07 mikelkew