trufflehog icon indicating copy to clipboard operation
trufflehog copied to clipboard

trufflehog git concurrency does not work as expected

Open yang0369 opened this issue 11 months ago • 3 comments

I am trying to speed up the trufflehog git scan with this "--concurrency" arg. After some trials, it does not look very right. 1 As shown, it takes 14s when concurrency=1 while 15s when concurrency=8 (given my desktop has 8 cores), which contradicts with common sense. Could someone shed some light on this please? Thanks.

yang0369 avatar Dec 16 '24 14:12 yang0369

Anyone can help with this? As I am testing with trufflehog git repo, so it can be easily replicated with the below code:

trufflehog git --json --concurrency=8 https://github.com/trufflesecurity/trufflehog.git --max-depth=6 --no-verification trufflehog git --json --concurrency=1 https://github.com/trufflesecurity/trufflehog.git --max-depth=6 --no-verification

all the ideas are welcomed and much appreciated!

yang0369 avatar Dec 24 '24 01:12 yang0369

I've also noticed strange behavior with the --concurrency switch. Even with the flag set to --concurrency=1, I can see more than one thread in the process list.

Image

enp-mrygiel avatar Jan 23 '25 10:01 enp-mrygiel

I've also noticed strange behavior with the --concurrency switch. Even with the flag set to --concurrency=1, I can see more than one thread in the process list.

Image

I am also wondering about this.

I'm trying to use the --concurrency option to ensure that trufflehog sends data output to the Verification Server in an orderly fashion. For example, when scanning a folder with multiple files, I expected that setting --concurrency=1 would make trufflehog scan one file at a time and send the result sequentially to the Verification Server. However, it seems that even with --concurrency=1, the tool attempts to send data concurrently using multiple threads.

Could someone verify whether this behavior is expected or if it might be a bug? Thank you in advance to the developers.

jjang3 avatar Feb 18 '25 18:02 jjang3