usnistgov
Generated Test vectors for RSA PSS with MGF1=SHAKE128 uses 16 byte output length for MGF1.
There are test vectors in https://github.com/usnistgov/ACVP-Server/blob/master/gen-val/json-files/RSA-SigGen-FIPS186-5/internalProjection.json For the combination of hashAlg=SHAKE128 MGF=MGF1
The signature was only verified when the output size for SHAKE 128 is 32 for the HashAlg and 16 during the MGF. I was expecting the same output length of 32 for both operations? (16 bytes gives only half of the security strength)
Is this mismatch deliberate?, or has this been fixed and new test vectors should be generated?
I am starting to think that using SHAKE inside MGF1 should not be allowed.
In FIPS 186-5 5.4 (b) Specifically states that if "Hash" is SHAKE then the output size MUST BE 32/64 bytes for SHAKE128/256 5.4.1 Mask Generation Functions in RSASSA-PSS allows SHAKE as alternative MGF's. It does not however mention using shake inside MGF1, or what the default size should be.
Using an XOF as a MGF makes sense to me, but using it as a general hash seems much less useful if a fixed output length is required (such as inside MGF1).
RFC 8702 unambiguously states that when SHAKE is used in digital signature hash, SHAKE must be used as the MGF function, and not as mgf1(shake). Thus half of the shake based test vecors in the demo ACVP server appears to be in violation of RFC 8702.
Also FIPS 186-5 explicitely allows SHAKE of certain lengths to be used as message digests; and explicitely allows to use SHAKE as an MGF; and makes no comment as to whether or not shake is allowed to be used inside MGF1, and if so which size it should use.
In the https://github.com/usnistgov/ACVP-Server/blob/master/gen-val/json-files/RSA-SigVer-FIPS186-5/internalProjection.json the projections that have
"hashAlg": "SHAKE-128" with a matching "maskFunction": "shake-128" (and ditto matching shake-256) appear to be compliant with a narrow interpretation of FIPS 186-5 and also RFC 8702 compliant.
However those that use "hashAlg": "SHAKE-256" with "maskFunction": "mgf1" appear to violate RFC 8702 and are not explicitely defined in FIPS 186-5. Without an explicit mention in the FIPS 186-5 or implementation guide, it would be best to not offer such a combination in the ACVP demo server. As it violates RFC 8702 and not defined explicitely by FIPS 186-5. (as only explicitely allowed hash functions are defined for MGF1 or shake-as-MGF-but-not-in-MGF1 is defined).
If hashAlg SHAKE is desired to be tested with maskFunction mgf1, one needs to somehow specify the mgf1-hashAlg i.e. such that SHAKE of a fixed length is used as message digest; and MGF1 is used with an approved hash...... which would appear to be inline with FIPS 186-5, yet still violate RFC 8702.
Whilst all of the above is undefined, or explicitly prohibited by some standards bodies. It would be best if ACVP demo server would not have any combinations of "hashAlg shake with maskFunction mgf1", or even better explicitly prohibit requesting that.
The combinations of "hashAlg shake with maskFuction shake" are valid and should remain.
@slontis, sorry, it looks like this ticket fell through the cracks.
There are test vectors in https://github.com/usnistgov/ACVP-Server/blob/master/gen-val/json-files/RSA-SigGen-FIPS186-5/internalProjection.json For the combination of hashAlg=SHAKE128 MGF=MGF1
The signature was only verified when the output size for SHAKE 128 is 32 for the HashAlg and 16 during the MGF. I was expecting the same output length of 32 for both operations? (16 bytes gives only half of the security strength)
Is this mismatch deliberate?, or has this been fixed and new test vectors should be generated?
The same behavior was reported in #277. The difference between #277 and what you reported is that in #277 the user was requesting test vectors from the ACVTS Demo server. We addressed the issue on our servers, but it looks like we didn't update the sample test vectors until earlier this month.
@slontis and @xnox, I see your additional comments. We'll take a look and get back to you.
Best,
Ben
@livebe01 any updates on this?
Should I open a different issue? Many test cases are listed as passing -> yet they use settings that are invalid as per RFC and FIPS 186-5. Thus likely going to lead to interoperability issues.
I guess, I should test things against demo server to confirm server behaviour versus the static tests.