terraform-provider-gsuite
terraform-provider-gsuite copied to clipboard
DeviaVir
Set custom quota user to avoid throttling
The admin SDK supports setting arbitrary quotaUser values: https://developers.google.com/admin-sdk/directory/v1/parameters
If this isn't set, the consumer IP will be used which can lead to throttling.
Updating the provider to use a custom/random quota user on each request would reduce throttling.
Hey @morgante, it's unfortunately not really clear to me how to use this with the oauth clients in golang used in this provider. Any suggestions or links? I tried searching the web but only found Java results.
Unfortunately I'm not familiar with the Go SDK but I'm guessing it might be a flag where you set the access token?
I believe that would have to happen somewhere here: https://github.com/DeviaVir/terraform-provider-gsuite/blob/master/gsuite/config.go#L49-L100 but at this time I'm unsure of the change that is required to make that work, it'd seem like oauth would need to accept another query parameter?
I don't know if it's the throttling, but the plans are taking much longer (10x) after upgrading.
Edit: I realized the issue for the slowness is the retry on 404s. Why are you retrying on 404s? Sounds wrong.
Off-topic: answered this question elsewhere too, but will answer it here as well: the 404's are necessary to have 100% successful applies. Sometimes it takes Gsuite a while before a resource is created, while it does it will return 404's and the apply will fail. If we retry the 404's a few times we have a success rate of 100% for applies, which is quite important for automation (jenkins). If anything we could try finetuning what and when to retry.
The quotauser can be used if you decide to apply a cap, this is an Opt in (from what i understand from the docs).
The page OP mentions also links to the capping instructions which explain how to enable this.
Other than that the usage limits are per API (also mentioned on the capping page) so unless you have a script spamming the Directory API I don't expect anyone to hit this limit / needing this feature.
So.. time to close this issue?
@TheWolfNL It is possible to set caps yourself, but Google also enforces certain caps out of the box (per IP) to prevent abuse.
This provider can end up making a lot of requests to the Directory API if you're creating multiple groups and managing their membership.
I can attest to the fact this provider can easily hit quota limits on the Directory API if you control a lot of groups. Wondering if quotauser needs to be set on every request, and if we can set it to the group email while refreshing...