[BUG][Functional] Security Hub Disable Standards Controls - TooManyRequestsException

[para0056]

Required Basic Info

• Accelerator Version: (eg. v1.3.8-a)
• Install Type: N/A
• Upgrade from version: N/A

Describe the bug Attempted to disable multiple Security Hub Standards Controls (19 controls in the Foundational Security Best Practices Framework and 19 controls in the CIS Benchmark) by adding the controls to the controls-to-disable array in the config.json. The State Machine completed successfully, but the Security Hub console does not show the controls as "Disabled" (even after waiting the requisite +24h). Instead, according to the Security Hub console, the controls have a status of "No data".

Failure Info Based on CloudTrail data, the Security-Phase4-CustomSecurityHubDisable Lambda hit a number of TooManyRequestsException errors when calling the UpdateStandardsControl API to disable the standards controls. Unfortunately, the Lambda seems to have effectively swallowed that error and did not pass it back to the StateMachine. We attempted to run the State Machine again, but Security-Phase4-CustomSecurityHubDisable did not get executed.

Steps To Reproduce

1. Modify the security-hub-frameworks -> standards -> controls-to-disable array and add the controls that you wish to disable.
2. Run the Main State Machine

Expected behavior

I expected the ASEA to disable the controls as specified in the config.json file. Or at the very least, not swallow the throttling exception.

Screenshots

Additional context

We had tested this method of disabling multiple Security Hub controls in our lower-state ASEA environment (same ASEA version) and did not encounter this issue. However, the lower state environment only has 11 accounts, whereas our proper environment (where we encountered the bug) has 20 accounts. More accounts = more API calls to Security Hub...?

Another oddity - the Security Hub console displays these "bugged" standards controls with the status of "No data", while the CLI/API return a status of "DISABLED". We've opened Support case and engaged our TAM to explain/address this discrepancy.

[rjjaegeraws]

Hi para0056,

The SM will actually 'retry' the CDK/CloudFormation deployment 3 times if there is an error. It's possible that you are seeing an entry for throttling, that would have failed the deployment, but it succeeded in a subsequent attempt. If you do a search in CloudTrail for that eventName + region + account, you may see failed followed by successful entries.