Add more regex detection for X.509/OpenSSH/OpenPGP materials

[pmevzek-godaddy]

Feature Request

Is your feature request related to a problem? Please describe.

Technically this is related to https://github.com/dxa4481/truffleHogRegexes/blob/master/truffleHogRegexes/regexes.json which is in another repository but maybe because of https://github.com/godaddy/tartufo/issues/5 here it makes more sense to report it here. But I guess it can be moved if deemed necessary.

The content potentially found by the added expressions would nevertheless normally also be found by the high entropy scanners due to the content, but it may be better to add the expressions nevertheless.

Side note: at least for the X.509 case, but maybe others, the PEM encoded format (which is base64 inside header+trailer) is not the only possible case of storing keys/certificates. The binary encoded format (DER) could also happen and hence the content wouldn't be found then, which can't be found by a regex, but a full X.509 parser would be needed, which may not be a small feat. Yet, it shows that secrets in binary are not so well detected as secrets in pure text, which may warrant at least a warning in documentation.

X.509

Private keys related to X.509 certificates are stored in PEM format in that way:

-----BEGIN PRIVATE KEY-----
... base64...
-----END PRIVATE KEY-----

Looking at openssl source file you can find also -----BEGIN ENCRYPTED PRIVATE KEY----- but not sure if an encrypted key should be flagged as secret.

There is a need to add a regex to match this header.

OpenSSH

A search for BEGIN in their repository at https://github.com/openssh/openssh-portable/search?q=BEGIN yields at least the following results of possible header to search for:

• -----BEGIN OPENSSH PRIVATE KEY----- (this happens for example with specific "new" OpenSSH key formats like sk-[email protected])
• ---- BEGIN SSH2 ENCRYPTED PRIVATE KEY ---- (dubious, if the key is encrypted, should it be reported as a potential secret?)

There are also other kind of similar headers for signatures or public keys, but I guess those should not be reported as secrets.

OpenPGP

A search in their repository yields the following header cases that should probably be considered as hinting the presence of a secret:

• -----BEGIN PGP SECRET KEY BLOCK-----
• -----BEGIN OPENSSH PRIVATE KEY----- (also in previous section)

Describe the solution you'd like

Add some more regex to cover the above examples.

Describe alternatives you've considered

The high entropy scanner is also able to spot the strings, so it is a safeguard. But it may not be chosen during run, so it would be better to improve the regex list.

Teachability, Documentation, Adoption, Migration Strategy

Currently there is a single list of regex applied, so a change in it would automatically apply to all users. Which is another reason why https://github.com/godaddy/tartufo/issues/176 could be useful to have so one can know for a given set of results which precise checks (regexs) were applied.

[tarkatronic]

@pmevzek-godaddy With #5 fixed, the default patterns now live within the tartufo codebase itself. Are these patterns something you would be willing to contribute? They live here now: https://github.com/godaddy/tartufo/blob/main/tartufo/data/default_regexes.json