re2 error: `re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35`

[rgmz]

Please review the Community Note before submitting

TruffleHog Version

Trace Output

re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35
re2/re2.cc:772: DFA out of memory: pattern length 102, program size 928, list count 352, bytemap range 35

Expected Behavior

The chunk data should be scanned.

Actual Behavior

TruffleHog outputs the aforementioned error from re2, making it unclear what the cause is and whether certain chunks were skipped.

Steps to Reproduce

The error seems semi-random so it's difficult to reproduce. Additionally, the log comes directly from re2.cc, meaning there is no context associated with it.

Environment

• OS: [e.g. iOS]
• Version [e.g. 22]

Additional Context

https://github.com/google/re2/issues/186

References

• #0000

[zricethezav]

Maybe providing an option for users to pick which regex engine they want, re2 or default, would be worthwhile since re2 is a drop-in replacement of regex

[rgmz]

It couldn't hurt given #2354. I think this specific error is caused by the configured max_mem for re2 being smaller than the TruffleHog's maximum diff size.

https://github.com/trufflesecurity/trufflehog/blob/2888f8cdfcb1b70f1814dc223d17d45fc4eebb20/pkg/gitparse/gitparse.go#L27-L28