trufflehog
trufflehog copied to clipboard
AWS credentials in files not detected in git scan
Please review the Community Note before submitting
TruffleHog Version
3.72.0
Trace Output
Expected Behaviour
It should detect AWS credentials saved in files.
Actual Behavior
It doesn't detect them at all.
Steps to Reproduce
- In a repository, create the
.aws/credentials
file and add the AWS creds like:
[default]
aws_access_key_id=****
aws_secret_access_key=****
- Run
trufflehog filesystem .
- Below logs output is produced:
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
2024-03-25T17:16:40Z info-0 trufflehog running source {"source_manager_worker_id": "YnfOv", "with_units": true}
2024-03-25T17:16:40Z info-0 trufflehog finished scanning {"chunks": 696, "bytes": 660010, "verified_secrets": 0, "unverified_secrets": 0, "scan_duration": "158.160668ms"}
- It doesn't detect anything.
Environment
- OS: MacOS
- Version [Sonoma 14.4]
Additional Context
References
- #0000
Not sure if I'm having the same issue as @Constantin07 but adding to this since I'm also scanning AWS secrets in the filesystem and I can't quite understand the inconsistency with this tool.
I created an example repository and it doesn't pick up the secrets in the file or in the Git objects (it does in another repo).
I'm running a different version through docker:
docker run --platform linux/arm64 -v $PWD:/pwd trufflesecurity/trufflehog:latest --version
trufflehog 3.68.4
The following is the command I run, both in the repo it detects the AWS secret properly and the repo included here
docker run --platform linux/arm64 -v $PWD:/pwd trufflesecurity/trufflehog:latest filesystem /pwd
I don't quite understand what gets trufflehog to finally detect properly.
This comment may also be related.
Out of curiosity, does this occur if you run trufflehog filesystem . --allow-verification-overlap
?
It may be a similar issue as #2515; if not, I could try to modify the script debugging script in https://github.com/trufflesecurity/trufflehog/issues/2515#issuecomment-1967072147 to see if there's a particular commit responsible.
Hey @rgmz, same unexpected result with that command.
I tried to find a good commit to start from and I couldn't find one, even back 1-2 years ago. I took the same approach as your bisect script but without the run, building the Docker image and running it against my repo. I was hoping I'd find something that way I could share it with you, but nothing.
I also double checked that the current AWS regex matches the fake AWS creds in that repo and it does.
Maybe you'll have better luck.
Ah, I took a look at your repo and noticed that your key ("AKIAIOSFODNN7EXAMPLE") is an obvious false-positive. This results in it being filtered by the false-positive check.
https://github.com/trufflesecurity/trufflehog/blob/becce51335f98090b9729b00a740aed3bac16664/pkg/detectors/aws/aws.go#L231-L235
https://github.com/trufflesecurity/trufflehog/blob/becce51335f98090b9729b00a740aed3bac16664/pkg/detectors/words.txt#L22
I'm not sure if @Constantin07's issue is similar, or if the credentials in their example are 'live'.
@rgmz yes, I've used live credentials.
Hmm, strange. What do you see if you run the following commands (based on this branch)? (This will print the ID/Secret/account number, so don't run it somewhere that it can be exposed in the logs, and remove that information if you paste the output.)
For the example that Mike shared, we can see that it skips the result because of the false-positive check.
git clone [email protected]:rgmz/trufflehog.git
cd trufflehog
git switch -C debug/aws-logging
go build
./trufflehog filesystem /path/to/repo
# Output
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
2024-04-02T14:05:04-04:00 info-0 trufflehog possible duplicate detector configured {"detector": "JiraToken.v2"}
2024-04-02T14:05:04-04:00 info-0 trufflehog running source {"source_manager_worker_id": "Hx87Z", "with_units": true}
2024-04-02T14:05:04-04:00 info-0 trufflehog.aws Found ID {"detector_worker_id": "usiWR", "timeout": 10, "id": ["AKIAIOSFODNN7EXAMPLE", "AKIAIOSFODNN7EXAMPLE", "AKIA"]}
2024-04-02T14:05:04-04:00 info-0 trufflehog.aws Found ID {"detector_worker_id": "4aQdJ", "timeout": 10, "id": ["AKIAIOSFODNN7EXAMPLE", "AKIAIOSFODNN7EXAMPLE", "AKIA"]}
2024-04-02T14:05:04-04:00 info-0 trufflehog.aws Found Secret {"detector_worker_id": "4aQdJ", "timeout": 10, "secret": ["\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\"", "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"]}
2024-04-02T14:05:04-04:00 info-0 trufflehog.aws Determined account number {"detector_worker_id": "4aQdJ", "timeout": 10, "account": ""}
2024-04-02T14:05:04-04:00 info-0 trufflehog.aws Attempting to verify match {"detector_worker_id": "4aQdJ", "timeout": 10}
2024-04-02T14:05:04-04:00 info-0 trufflehog.aws Skipping result, it is a false-positive {"detector_worker_id": "4aQdJ", "timeout": 10}
2024-04-02T14:05:04-04:00 info-0 trufflehog finished scanning {"chunks": 27, "bytes": 26566, "verified_secrets": 0, "unverified_secrets": 0, "scan_duration": "411.535354ms"}
(If at any point my issue is confirmed separate from Constantin07 I'll create a new issue, for now I'll just continue here)
I took an active AWS IAM cred, put it into the code, and modified the secret access key slightly to see if it would trigger. It did not. I even committed it to give it a bit more area to catch it, didn't catch it.
Here was the output on main
branch
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
2024-04-02T22:38:20Z info-0 trufflehog possible duplicate detector configured {"detector": "JiraToken.v2"}
2024-04-02T22:38:20Z info-0 trufflehog running source {"source_manager_worker_id": "rDI2H", "with_units": true}
2024-04-02T22:38:21Z info-0 trufflehog finished scanning {"chunks": 19, "bytes": 24222, "verified_secrets": 0, "unverified_secrets": 0, "scan_duration": "629.643375ms"}
I checked out your branch, built it with Docker. Here is the output (with .git/objects removed) with "REDACTED" where the id/secret or account id would be:
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
2024-04-02T22:35:01Z info-0 trufflehog possible duplicate detector configured {"detector": "JiraToken.v2"}
2024-04-02T22:35:01Z info-0 trufflehog running source {"source_manager_worker_id": "rxKlR", "with_units": true}
2024-04-02T22:35:01Z info-0 trufflehog.aws Found ID {"detector_worker_id": "Rplh1", "timeout": 10, "id": ["REDACTED", "REDACTED", "AKIA"]}
2024-04-02T22:35:01Z info-0 trufflehog.aws Found Secret {"detector_worker_id": "Rplh1", "timeout": 10, "secret": ["\"REDACTED\"", "REDACTED"]}
2024-04-02T22:35:01Z info-0 trufflehog.aws Determined account number {"detector_worker_id": "Rplh1", "timeout": 10, "account": "REDACTED"}
2024-04-02T22:35:01Z info-0 trufflehog.aws Attempting to verify match {"detector_worker_id": "Rplh1", "timeout": 10}
2024-04-02T22:35:01Z info-0 trufflehog.aws Found ID {"detector_worker_id": "wg89M", "timeout": 10, "id": ["REDACTED", "REDACTED", "AKIA"]}
2024-04-02T22:35:02Z info-0 trufflehog.aws Skipping result, it is a false-positive {"detector_worker_id": "Rplh1", "timeout": 10}
2024-04-02T22:35:02Z info-0 trufflehog finished scanning {"chunks": 19, "bytes": 24222, "verified_secrets": 0, "unverified_secrets": 0, "scan_duration": "472.759083ms"}
I took an active AWS IAM cred, put it into the code, and modified the secret access key slightly to see if it would trigger. It did not. I even committed it to give it a bit more area to catch it, didn't catch it. ... 2024-04-02T22:35:02Z info-0 trufflehog.aws Skipping result, it is a false-positive {"detector_worker_id": "Rplh1", "timeout": 10}
Interesting, that definitely shouldn't be the case.
Can you pull the latest changes and try again? I added some print statements which should show the specific reason it's getting flagged as a false-positive.
2024-04-02T19:03:13-04:00 info-0 trufflehog.aws Attempting to verify match {"detector_worker_id": "V5ZP0", "timeout": 10}
matched FalsePositive 'example'
2024-04-02T19:03:13-04:00 info-0 trufflehog.aws Skipping result, it is a false-positive {"detector_worker_id": "V5ZP0", "timeout": 10}
Okay, I know why for me it didn't flag a real access key id. When I modified the access secret I left the EXAMPLE
portion in the secret, when I changed it to EZAMPLE
it flags as expected. I now see that that word is in the code base to ignore.
Anyways, sorry for hijacking this @Constantin07. It's possibly not the same issue as you. Thanks @rgmz for bearing with me.
I have generated a new set of AWS credentials and now it does detect them in filesystem scan:
trufflehog filesystem .
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
2024-04-03T11:07:21+01:00 info-0 trufflehog running source {"source_manager_worker_id": "0RByN", "with_units": true}
Found unverified result 🐷🔑❓
Detector Type: AWS
Decoder Type: PLAIN
Raw result: AKIAU6GDVF*XE1N*****
Resource_type: Access key
Account: *******************
File: .aws/credentials
Line: 1
Found unverified result 🐷🔑❓
Detector Type: AWS
Decoder Type: PLAIN
Raw result: AKIAU6GDVF*XE1N*****
Resource_type: Access key
Account: 339712749422
File: .git/objects/93/78557dddb3ad9b9492f2b3f71e8851e1858df3
Line: 1
But detection doesn't work when I scan the repository where the commit has been made with credentials (it's in commit log).
Here is the pre-commit config:
- repo: local
hooks:
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data with TruffleHog.
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail --no-update'
language: system
stages: [pre-commit, pre-push, manual]
(Following example from here https://github.com/trufflesecurity/trufflehog?tab=readme-ov-file#pre-commit-hook)
pre-commit run -a
...
TruffleHog...............................................................Passed
Manually:
$ trufflehog git file://. --since-commit HEAD --only-verified --fail --no-update
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
2024-04-03T11:17:17+01:00 info-0 trufflehog running source {"source_manager_worker_id": "EcnaA", "with_units": true}
2024-04-03T11:17:17+01:00 info-0 trufflehog finished scanning {"chunks": 0, "bytes": 0, "verified_secrets": 0, "unverified_secrets": 0, "scan_duration": "72.81015ms"}
If I rase a PR in GH with a branch containing AWS secrets, the below GH action check doesn't detect anything either:
GH action check:
jobs:
...
secrets-scan:
runs-on: ubuntu-22.04
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Secret Scanning
uses: trufflesecurity/trufflehog@main
with:
extra_args: --only-verified
(Following example from here https://github.com/trufflesecurity/trufflehog?tab=readme-ov-file#general-usage)
Output:
I was expecting it to scan git history for diff between PR branch and the main
one for leaked creds but it doesn't work for some reason despite PR contains valid AWS creds.
So filesystem scan works but git scan doesn't. What am I doing wrong ?