AWS credentials issue

[za]

Hi,

I am trying to run this and having this ERROR:

$ docker-compose run \
    -e AWS_PROFILE=default \
    -e AWS_DEFAULT_REGION=us-east-1 \
    cartography --neo4j-uri bolt://cartography-neo4j-1:7687

INFO:cartography.sync:Starting sync stage 'aws'
ERROR:cartography.intel.aws:Unable to initialize the default AWS session, an error occurred: The config profile (default) could not be found. Make sure your AWS credentials are configured correctly, your AWS config file is valid, and your credentials have the SecurityAudit policy attached.
INFO:cartography.sync:Finishing sync stage 'aws'

I have configured my aws credentials correctly:

$ aws sts get-caller-identity

returns my AWS user account detail. Where did I miss?

[achantavy]

Hi there, what does your ~/.aws/config look like, and how are you specifying your credentials? See also https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials. AWS creds can be a bit complicated.

I would also try the following permutations for cartography:

1. Don't specify AWS_PROFILE
2. Don't specify AWS_PROFILE and do specify --aws-sync-all-profiles

Happy to get on a call if that helps, reach out to me on Slack

[za]

Hi @achantavy

I am defining my access/secret key on .aws/credentials and this is my .aws/config:

[default]
region=<snipped>

OK, opening slack now...

[za]

BTW, which aws CLI version does this cartography using @achantavy ? Reference on this page https://cartography-cncf.github.io/cartography/modules/aws/config.html is linking to v1.

[za]

Happy to get on a call if that helps, reach out to me on Slack

Hi @achantavy where's the slack address? I couldn't find it.

[za]

Happy to get on a call if that helps, reach out to me on Slack

Hi @achantavy where's the slack address? I couldn't find it.

Found it https://communityinviter.com/apps/cloud-native/cncf but it returns 522

[achantavy]

Huh, that's weird, communityinviter has been down for a while it seems, I'll see if I can ping cncf

[achantavy]

I am defining my access/secret key on .aws/credentials and this is my .aws/config:

Oh, I see it now! Your docker-compose command is not passing any credentials to the docker container!

So the command you ran here (https://github.com/cartography-cncf/cartography/issues/1487#issue-2919436952) does not pass any files or creds to the docker container.

You need map your aws files on your host as a volume to the docker container for the credentials to be discovered by docker. Do it like this with the -v:

$ docker-compose run \
    -e AWS_PROFILE=default \
    -e AWS_DEFAULT_REGION=us-east-1 \
     -v ~/.aws:/var/cartography/.aws/ \
    cartography --neo4j-uri bolt://cartography-neo4j-1:7687

Side note: Looks like communityinviter is back up again, look forward to seeing you on slack

Additional reference: Cartography uses the boto3 python library to grab AWS data, and boto3 discovers credentials in this order: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials, so if something seems to be conflicting, you can consult this doc

[achantavy]

@za --^

[achantavy]

Hey @za , how's this been going? I think the main issue is making sure the creds are passed correctly to the container. If the docker stuff doesn't work out, you can try run it with python with a venv. Either way I'm happy to walk you through it, ping me on Slack and we can find a time

[za]

Hi @achantavy Thanks for pinging. IIRC, I haven't managed to solve the issue. I'll find you on slack...