Secrets Manager backend does not support partial ARNs

[enezhadian]

The moto secrets manager backend is unable to retrieve secrets when a partial ARN is passed as secret ID.

How to reproduce the issue Here is a minimal example that demonstrates the issue:

import boto3
import pytest
from moto import mock_secretsmanager
from moto.core import DEFAULT_ACCOUNT_ID


@pytest.fixture
def secretsmanager():
    return boto3.client("secretsmanager", region_name="us-east-1")


@pytest.fixture
def secret(secretsmanager):
     with mock_secretsmanager():
        name = "test-secret"
        secretsmanager.create_secret(Name=name, SecretString="42")
        yield name


def test_retrieval_using_name(secretsmanager, secret):
    secretsmanager.get_secret_value(SecretId=secret)


def test_retrieval_using_partial_arn(secretsmanager, secret):
    partial_arn = f"arn:aws:secretsmanager:us-east-1:{DEFAULT_ACCOUNT_ID}:secret:{secret}"
    secretsmanager.get_secret_value(SecretId=partial_arn)

What is expected to happen In this code snippet, it is expected that both tests pass, i.e. the secret can be retrieve using secret name as well as the secret partial arn (ARN exlcuding the random suffix).

What actually happens The test_retrieval_using_partial_arn test fails with the following error:

botocore.errorfactory.ResourceNotFoundException: An error occurred (ResourceNotFoundException) when calling the GetSecretValue operation: Secrets Manager can't find the specified secret.

What version of Moto you're using moto version: 4.0.7 boto3 version: 1.24.90 botocore version: 1.27.90

Root cause The issue seems to originate in get_secret_name_from_arn, which is called when the secret_id is validated to exist. get_secret_name_from_arn will turn a partial ARN into "test" instead of "test-secret".

[bblommers]

From the docs (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.create_secret):

Do not end your secret name with a hyphen followed by six characters. If you do so, you risk confusion and unexpected results when searching for a secret by partial ARN. Secrets Manager automatically adds a hyphen and six random characters after the secret name at the end of the ARN.

AWS does allow this however. When asking AWS for a secret that ends with a hyphen followed by six characters it looks like it will try to do an exact match first. Given two secrets created in AWS:

aws secretsmanager list-secrets
{
    "SecretList": [
        {
            "ARN": "arn:aws:secretsmanager:us-east-1:0000:secret:test-secret-CBDWr2",
            "Name": "test-secret",
           ...
        },
        {
            "ARN": "arn:aws:secretsmanager:us-east-1:0000:secret:test-secret-CBDWr2-GOgMjk",
            "Name": "test-secret-CBDWr2",
           ...
        }
    ]
}

Searches by name:

aws secretsmanager describe-secret --secret-id test-secret
{
    "ARN": "arn:aws:secretsmanager:us-east-1:0000:secret:test-secret-CBDWr2",
    "Name": "test-secret",
    ....
}

aws secretsmanager describe-secret --secret-id test-secret-CBDWr2
{
    "ARN": "arn:aws:secretsmanager:us-east-1:0000:secret:test-secret-CBDWr2-GOgMjk",
    "Name": "test-secret-CBDWr2",
    ...
}

Searches by ARN have to be an exact match:

aws secretsmanager describe-secret --secret-id "arn:aws:secretsmanager:us-east-1:0000:secret:test-secret"

An error occurred (ResourceNotFoundException) when calling the DescribeSecret operation: Secrets Manager can't find the specified secret.

aws secretsmanager describe-secret --secret-id "arn:aws:secretsmanager:us-east-1:0000:secret:test-secret-CBDWr2"
{
    "ARN": "arn:aws:secretsmanager:us-east-1:0000:secret:test-secret-CBDWr2",
    "Name": "test-secret",
    ....
}

TLDR: Marking it as an enhancement to support this scenario.

Thanks for raising this @enezhadian!

[enezhadian]

Thanks @bblommers for your comment 🙏 That's a good point and I think that would be a very useful enhancement. However the issue occurs regardless of that, and indeed, I haven't chosen an appropriate secret name in the code example. If you replace that secet name with anything else (such as testsecret), same error pops up. The problem lies in the way SecretsStore validates the existence of a secret_id. It first convert the secret_id into a secret name and then checks whether a secret with that name exists. To do that it calls get_secret_name_from_arn on the input secret_id, and when the secret_id is an ARN it removes the last part of the secret name after the last hyphen. However it doesn't take into account that the ARN might be a partial ARN which could result in a part of the secret name to be thrown away. I believe to resolve this issue the SecretsStore.__contains__ needs to be updated to account for partial ARNs.

If it's okay with you, I can open a PR with my proposed solution. What do you think?

[bblommers]

Ah, I see. A PR would be very welcome - it sounds like that is indeed the fix!

Would be good to have some tests for this as well - there are quite a few (in tests/test_secretsmanagers/) that you can use as inspiration. Let me know if you need any help with this.

[bblommers]

Hi @thoroc, just checking on this - is this still on your radar?

If you're stuck, or don't have any time to finish up, just let me know - I'd be happy to have a go at it as well.