trufflehog 3.68.2 does not detect a github PAT detected by 3.63.4

[jingles]

TruffleHog Version

3.68.2

Trace Output

https://gist.github.com/jingles/262a42a6edaf0647cf723ceb6ce09f51

Expected Behavior

Should have detected verified Github result

Actual Behavior

no results found

Steps to Reproduce

see gist, these are private repos

Environment

• Dockerized

Additional Context

I was intentionally testing exposure of a Github PAT in github actions and trufflehog failed to detect it. locally it worked. Eventually I realized my local docker container was behind the latest pulled in the actions build. Updated my image locally and the same command failed to detect.

[rgmz]

Is the PAT the old format or the new one with the prefix (ghp_xxxx)? 🤔

[jingles]

new limited scope format with github_pat_ prefix...

[jingles]

wonder if this could relate to https://github.com/trufflesecurity/trufflehog/pull/2324/files ??

[rgmz]

Are you able to reproduce this locally (i.e., not on GitHub Actions)? If so, it should be easy to determine which commit caused the issue with git bisect.

[jingles]

i can reproduce locally using dockerized trufflehog - do you have any additional info or guide on this tool? do i need to install golang (it's been a while) so i can compile and run each version? thanks

[rgmz]

do i need to install golang (it's been a while) so i can compile and run each version?

You could try to do it with Docker.

Steps

(This will likely require some tweaks to get working.)

1. Create the following script somewhere (I'm putting this in /tmp/bisect.sh).

# Write the script
cat << 'EOF' > /tmp/bisect.sh
#!/usr/bin/env bash

# an exit code of 125 asks "git bisect" to "skip" the current commit
IMAGE=$(docker buildx build --platform=linux/amd64 -q . || exit 125)

# run the application and check that it produces good output
docker run --rm -v "$REPO_PATH":/tmp -w /tmp -it "$IMAGE" git file:///tmp/ --since-commit ${base_commit:-''} --branch ${head_commit:-''} --fail --no-update --github-actions --only-verified

if [ $? -eq 183 ]; then
    exit 0
else
    exit 1
fi
EOF

# Make the script executable
chmod +x /tmp/bisect.sh

2. Clone your desired repo somewhere and set REPO_PATH

git clone {your_repo}
export REPO_PATH="/your/path"

3. Clone trufflehog:

git clone https://github.com/trufflesecurity/trufflehog.git
cd trufflehog

4. Start and run the bisect

git bisect start
# Marks the last known 'good' commit as v3.63.4 (https://github.com/trufflesecurity/trufflehog/releases/tag/v3.63.4)
git bisect good b0fab16ad4b5cc42c38ee4e2ee362aa3c47dea24
# Marks the current HEAD as bad.
git bisect bad
git bisect run /tmp/bisect.sh

[jingles]

wow, thanks @rgmz, that's an amazing tool.

assuming i did this right the result is:

b2074ad05d6cff313a83b4f5bebf541a1d594dd7 is the first bad commit

(I can post the whole thing if you want)

this seems to be https://github.com/trufflesecurity/trufflehog/pull/2356

From this I glean i could possibly apply the --allow-verification-overlap flag, i will try that and see if it changes anything...

What are the other next steps?

Thank you!

[jingles]

Ok, I have verified that adding that flag does indeed cause my results to appear with the latest version... I did have the same secret in a couple of different files, so perhaps this is the intended effect? But they are the same detector, not overlapping detectors... doesn't seem like a very good feature imho...

I guess it's up to you what to do with this information; thank you for the help and for now I will add this flag.

[rgmz]

FYI @ahrav @dustin-decker

[rosecodym]

@jingles if you run the latest version without the --only-verified flag, do any other detectors flag the secret? (They'd show it as unverified, but they would still trigger the overlap detection.)

[jingles]

No, they do not, @rosecodym

[rosecodym]

Then we've got a real mystery on our hands. #2356 is supposed to disable verification if a secret is flagged by multiple detectors, but that doesn't sound like what you're experiencing here, since you're only seeing one detector flag this secret. @ahrav I wonder if there's a bug in the implementation that's causing this secret to get dropped unnecessarily.

[rosecodym]

oh I should tag @zricethezav too

[zricethezav]

@jingles @rgmz we have a blog post out now covering why this feature was introduced: https://trufflesecurity.com/blog/contributor-spotlight-helena-rosenzweig-and-assetnote-team

But they are the same detector, not overlapping detectors... doesn't seem like a very good feature imho...

There is likely some bug in the implementation.

Do you have a sample file or repo we can test with? If you provide some sample fake secrets we can replace them with our own real secrets to be verified.

[philipfoo]

Just adding here that I am also seeing the same bug also not using verification - the detector worked when the secret was prefixed with ghp_ but it fails to detect a fine-grained PAT that begins with github_pat_. For what it's worth, I am running the tool with a filesystem scan.

A fake secret that should be detected but isn't: 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

Edit: it's interesting because I can see that the above secret is a match for the regex specified here, but it does not work when running the trufflehog scan on my computer.

[ahrav]

I'll take a closer at this today. Thank you @jingles for opening the issues and @philipfoo for the example.

[ahrav]

hey @philipfoo would it be possible for you to share the file or directory you put the fake secret in? Testing that secret locally it appears to be detected, there might be additional context in the other data that could be causing us to drop the secret, but it's hard to tell. If you are unable to share the file, could you maybe move the secret to a different file and test it there. Thanks a lot.