fides
fides copied to clipboard
Integrate redis rate limiter with saas connectors
Is your feature request related to a specific problem?
Once a rate limiter has been implemented and configs exist, we will need to integrate connectors with the rate limiter.
Describe the solution you'd like
A rate_limit config should exist within connectors or saas connectors config:
"rate_limit_config" : [
{
"rate" : 100,
"period" : seconds,
"custom_key" : my_custom_key
},
{
"rate" : 500000,
"period" : days,
"custom_key" : my_custom_key2
}
]
Use limiter from #1189 to limit connector calls. Rate limiter should block with a given timeout.
Additional context
First stab at this will only support saas connectors