InfluxDB Rotation Multi-user - logically impossible to create secret

[nsaxelby]

I'm trying to invoke a lambda to create a InfluxDB allAccess token from the readonly admin username/password. I am invoking the lambda sample : SecretsManagerInfluxDBRotationMultiUser/lambda_function.py, last commit for that file: 0a243a482f5e60769d5a8ca99019915a4f1796d9

I am sending the following to the lambda to test the function:

{
  "SecretId": "arn:aws:secretsmanager:eu-west-1:12345678910:secret:my-secret",
  "ClientRequestToken": "my-secret-version-123456",
  "Step": "createSecret"
}

My secret (arn:aws:secretsmanager:eu-west-1:12345678910:secret:my-secret) has a version (my-secret-version-123456) with an AWSPENDING tag.

I want to call the lambda to generate a new InfluxDB token, and populate the secret version: my-secret-version-123456 with that token. There appears to be a conflict/unreachable path in the sample lambda code. It appears impossible to hit line 154.

The application flow is controlled by an exception which is caught at line 153:

https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/92f00b3e3b32df58a8a7c230773335f5846c74fd/SecretsManagerInfluxDBRotationMultiUser/lambda_function.py#L153-L154

The get_secret_dict on line 151 needs to throw the specific ResourceNotFoundException exception: https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/92f00b3e3b32df58a8a7c230773335f5846c74fd/SecretsManagerInfluxDBRotationMultiUser/lambda_function.py#L150-L152

Within the get_secret_dict method, the only way to throw this ResourceNotFoundException is on line 333:

https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/92f00b3e3b32df58a8a7c230773335f5846c74fd/SecretsManagerInfluxDBRotationMultiUser/lambda_function.py#L331-L334 The secrets managers needs to not find a secret for this arn, stage, and version. The problem is that there is validation logic prior to this code being reached which checks that this secret, stage and version is present: https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/92f00b3e3b32df58a8a7c230773335f5846c74fd/SecretsManagerInfluxDBRotationMultiUser/lambda_function.py#L93-L107 Line 99 ensures that the version is present in the secret. Line 102 ensures that the version does not have AWSCURRENT label. Line 105 ensures that the version provided to the lambda has the AWSPENDING label attached. is it therefore impossible to cause a ResourceNotFoundException for this secret, version and stage combination?

[jirkafajfr]

The problem is with the assumption in the beginning:

My secret (arn:aws:secretsmanager:eu-west-1:12345678910:secret:my-secret) has a version (my-secret-version-123456) with an AWSPENDING tag.

The AWSPENDING label is also used as mechanism to prevent two rotations from overlapping each other. If your secret already has this label the createSecret step is skipped (please read our documentation about idempotency). The step is skipped because lambda assumes that some other rotation is already running (otherwise the AWSPENDING would be removed by the finishSecret).

If you operate the steps manually, you should pass ClientRequestToken that points to the AWSCURRENT and make sure that secret has no label marked as AWSPENDING.

[jirkafajfr]

I am closing this ticket, please feel free to reopen it in case that you're not happy with the answer.

[ShivamZAMC]

Hi @jirkafajfr

If I pass ClientRequestToken that points to the AWSCURRENT then line 103 will log AWS secrets is already set as current and code will exit due to return statement.

The surprising part is that I put some logs to print the step and versions. I can see that when lambda is triggered for the first time at createSecret step the ClientRequestToken version has PENDING tag but it is still gets ResourceNotFoundException and goes into except block to execute put secret value.