Datastream rate limit

[mf-sky]

Terraform Version

Terraform v1.7.4 on darwin_arm64

• provider registry.terraform.io/akamai/akamai v5.6.0
• provider registry.terraform.io/hashicorp/aws v5.40.0
• provider registry.terraform.io/hashicorp/vault v3.15.2

Affected Resource(s)

• akamai_datastream

Terraform Configuration Files

provider "akamai" {
  edgerc         = "~/.edgerc"
  config_section = var.section
  request_limit  = 60 #seconds of padding between API requests to prevent 429s
}

Debug Output

│ Error: fetching stream information: API error: │ { │ "type": "", │ "title": "Failed to unmarshal error body. DataStream2 API failed. Check details for more information.", │ "detail": "Your request did not succeed as this operation has reached the limit for your account. Please try after 2024-03-18T19:57:45.045Z", │ "instance": "", │ "statusCode": 429, │ "errors": null │ }

Panic Output

N/A

Expected Behavior

No rate limit applyed

Actual Behavior

Got rate limited

Steps to Reproduce

Change multiple datastream resources. Execute a plan

Important Factoids

N/A

References

#311

this on pretty much describes the same. Ive applied the suggested fix with no success.

[PawelSnoch]

Hello @mf-sky ,

thank you for rising this ticket. Could you add more details about steps to reproduce? Does this error appear after terraform plan or apply?

Best regards, Pawel

[mf-sky]

Hi @PawelSnoch ! Thanks for the quick reply.

I have a terraform project with about 20 datastreams. The error appears on both.

Thanks!

[wzagrajcz]

Hi @mf-sky

We still were not able to reproduce your issue but we've noticed a documentation issue that may have caused your issue: In the config you mention request_limit as seconds of padding between API requests to prevent 429s (probably after provider documentation). Problem is that it is actually defined totally opposite: The maximum number of API requests to be made per second (0 for no limit) (taken from field description). With 20 datastreams and 60 requests per second, the 429s seems understandable.

Does adjusting config to actual definition of request_limit solves your issue? From our side we will make sure to adjust the documentation for this field.

Best regards, Wojciech

[mf-sky]

HI @wzagrajcz

Yes, i got that line from the docs. With 22 datastreams it fails with a request limit at 4 Reducing to something below that works. The default(no request_limit specified) is 0 right?

The API has the following "DataStream 2 limits each client to 20 requests per minute." Why would this fail with request_limit at 5, meaning 5 requests per second, if the limit is per minute? Because with 3, with the same amount of requests, it doesnt fail. Looks like the API as a limit per second also

Thanks, Manuel

[lkowalsk-akamai-com]

The documentation was misleading in that area (and is already corrected). The parameter specifies how many request per second will be done using terraform. With multiple datastreams it some calls can be done in parallel, but some will take longer to be processed hence the amount of requests will be smaller due to that. This request_limit is not required paramater, so defualt is to not provide it at all.

Please let us know if you have any more questions.