Improve Keccak support in hashlib including KangarooTwelve

[267d5092-5b30-456c-a804-baef2aaa5204]

BPO	39539
Nosy	@gpshead, @tiran, @Mariatta, @gvanas

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-02-03.10:25:05.173>
labels = ['type-feature', 'library', '3.9']
title = 'Improve Keccak support in hashlib including KangarooTwelve'
updated_at = <Date 2020-02-04.01:00:29.148>
user = 'https://github.com/gvanas'

bugs.python.org fields:

activity = <Date 2020-02-04.01:00:29.148>
actor = 'gregory.p.smith'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-02-03.10:25:05.173>
creator = 'gvanas'
dependencies = []
files = []
hgrepos = []
issue_num = 39539
keywords = []
message_count = 3.0
messages = ['361280', '361331', '361333']
nosy_count = 4.0
nosy_names = ['gregory.p.smith', 'christian.heimes', 'Mariatta', 'gvanas']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue39539'
versions = ['Python 3.9']

[267d5092-5b30-456c-a804-baef2aaa5204]

Dear all,

I think it would be nice if hashlib would include the support of Keccak with a chosen suffix, as well as the fast instance KangarooTwelve (K12).

1. Currently, hashlib's interface for Keccak only supports the 6 instances of FIPS 202 (SHA3-* and SHAKE*). However, the instances in NIST SP 800-185 (cSHAKE, KMAC, …) use a different suffix and therefore cannot be instantiated on top of the aforementioned 6 instances. Instead, simply adding the suffix as an argument to the constructor would enable a user to instantiate plain Keccak (as in Ethereum) or the SP 800-185 instances.

2. K12 is an alternative hash function (and XOF) in the Keccak family. It is fast, parallelizable and it benefits directly from the cryptanalysis on the (unchanged) underlying permutation since 2008. This would be IMHO a valuable addition to hashlib. Among others, implementations of K12 can be found in the XKCP on GitHub.

Kind regards, Gilles (co-designer of Keccak and K12)

[Mariatta]

Added hashlib module experts.

[gpshead]

Christian added the SHA3 support so is probably best to comment on this. The way our hashlib code is structured we always use the https://github.com/python/cpython/tree/master/Modules/_sha3 implementation rather than OpenSSL for these algorithms due to the better functionality of that Keccak code. So at first glance it should be reasonably straightforward to update it to accept an optional suffix constructor argument.

PR welcome.

[gpshead]

We've simplified our sha3 fallback code when OpenSSL doesn't provide it to be tiny_sha3 in https://github.com/python/cpython/issues/91254.

Supporting things outside of the very standardized hashes realm is better left to PyPI packages. See also our BLAKE3 stdlib rejection https://github.com/python/cpython/issues/83479#issuecomment-1093850686.

But your point 1 mentions the https://csrc.nist.gov/publications/detail/sp/800-185/final sha3 keccak variant hashes. Are these widely used by any system that is not a cryptocurrency? In any standard protocols? If so, they may be worth considering for hashlib.

[gpshead]

FYI - Our built-in hashlib SHA3/Keccak support now comes from the HACL* project when OpenSSL doesn't provide SHA3.

I'm closing this as there doesn't seem to be a compelling reason to add it vs a PyPI package for all of these special purpose less-common often non-standard variants.