Unexpected zero bits in SHAKE-128 VOT test output

[dspdon]

I'm seeing a possible problem in some message digest ("md") result strings recorded in SHAKE-128 ACVP test vector files. The problem appears in most of the VOT test cases (test group 3).

One of the test files is here: gen-val/json-files/SHAKE-128-1.0/internalProjection.json

In the VOT test cases, the last byte of md has unexpected, message-independent zero-value bits in the right-most N bit positions (LSbits), where N is from 1 to 4 bits. Specifically, N = min(8-R,R) where R = mdlen % 8. Only cases where R > 0 have these unexpected zero-bits, i.e., cases where mdlen is not a multiple of 8 bits. This implicates 453 of 512 VOT test cases. These unexpected zero-bits are in addition to the zero-bits that are expected in the left-most 8-R bit positions (MSbits) of the last byte when R > 0, which are normal zero-padding up to a full byte.

Aside from these 1 to 4 unexpected zero bits, all other bits in the md vector for these VOT test cases match the output of a test implementation I am using for SHAKE-128. Also, all other non-VOT test cases in the file match the output of that same test implementation. I suspect a bug in the generator code when computing the last byte of md in test group 3 when R > 0.

Example: tcId 1394 (the first VOT test case with R > 0) mdlen = 3381 R = mdlen % 8 = 5 N = min(8-R,R) = 3, the # of unexpected bits overwritten with 0 In the ACVP file, the last byte of md is 0x18 = b'00011000' You can see the right-most 3 LSbits are 0 (The left-most 3 MSbits are also 0, which is normal zero-padding to a full byte) An IUT gives 0x1F = b'00011111' for the last byte

I repeated the above to verify that N right-most zero-bits appear in the last byte of all 453 VOT test cases when R > 0, in addition to the expected 8-R left-most zero bits for padding.