airbyte icon indicating copy to clipboard operation
airbyte copied to clipboard

Allow a connector to set a "default sync cadence"

Open sherifnada opened this issue 3 years ago • 0 comments

Tell us about the problem you're trying to solve

Some source connectors have optimal settings for how often their syncs should run. For example:

  • Salesforce should run approximately once a day as its rate limit is computed daily i.e: if you consume the rate limit within 30 minutes, you need to wait 23.5 hours before the next sync will succeed
  • DB connectors using CDC should run as much as possible (ideally real time!)
  • Github connector's rate limits are bucketed by the hour, so it should run once an hour etc..

Describe the solution you’d like

I would like to have "default" sync cadences for a source connector (a connector should be able to opt-in to it). this way the user doesn't have to configure a cadence or have special knowledge of the workings of the API they're hitting to know how to set the right cadence.

sherifnada avatar Jan 07 '22 09:01 sherifnada