airbyte
airbyte copied to clipboard
Allow a connector to set a "default sync cadence"
Tell us about the problem you're trying to solve
Some source connectors have optimal settings for how often their syncs should run. For example:
- Salesforce should run approximately once a day as its rate limit is computed daily i.e: if you consume the rate limit within 30 minutes, you need to wait 23.5 hours before the next sync will succeed
- DB connectors using CDC should run as much as possible (ideally real time!)
- Github connector's rate limits are bucketed by the hour, so it should run once an hour etc..
Describe the solution you’d like
I would like to have "default" sync cadences for a source connector (a connector should be able to opt-in to it). this way the user doesn't have to configure a cadence or have special knowledge of the workings of the API they're hitting to know how to set the right cadence.