The RSA with key size 16 may provide an invalid key randomly

[BogdanGureev]

The RSA with key size 16 may provide an invalid key randomly. The generation itself does not report any errors but validation fails due to an invalid state. Worth to say I don't need such small-sized keys in production and this was revealed just while testing our integration.

OS: Windows 10 x64 Home 21H2 Version: Crypto++ 8.6.0 Build: Visual Studio 2015 Configuration: Debug x64 Runtime library: Multi-threaded Debug (/MTd) Test application: Simply added the Win32 console project to the crypttest solution and add the cryptlib.lib library as a dependency. Then build and run.

Test case:

#include <iostream>

#include "osrng.h"
#include "pssr.h"
#include "rsa.h"

int main()
{
	CryptoPP::RSA::PrivateKey privateKey;

	CryptoPP::AutoSeededRandomPool random;
	privateKey.GenerateRandomWithKeySize( random, 16 );

	const CryptoPP::Integer & value1{ privateKey.GetModulus() };
	const CryptoPP::Integer & value2{ privateKey.GetPublicExponent() };
	const CryptoPP::Integer & value3{ privateKey.GetPrivateExponent() };
	const CryptoPP::Integer & value4{ privateKey.GetPrime1() };
	const CryptoPP::Integer & value5{ privateKey.GetPrime2() };
	const CryptoPP::Integer & value6{ privateKey.GetModPrime1PrivateExponent() };
	const CryptoPP::Integer & value7{ privateKey.GetModPrime2PrivateExponent() };
	const CryptoPP::Integer & value8{ privateKey.GetMultiplicativeInverseOfPrime2ModPrime1() };

	auto pretty = []( const CryptoPP::Integer & i )
	{
		std::stringstream ss;
		ss << i;
		return ss.str();
	};
	std::cout << "value1: " << pretty( value1 ) << std::endl;
	std::cout << "value2: " << pretty( value2 ) << std::endl;
	std::cout << "value3: " << pretty( value3 ) << std::endl;
	std::cout << "value4: " << pretty( value4 ) << std::endl;
	std::cout << "value5: " << pretty( value5 ) << std::endl;
	std::cout << "value6: " << pretty( value6 ) << std::endl;
	std::cout << "value7: " << pretty( value7 ) << std::endl;
	std::cout << "value8: " << pretty( value8 ) << std::endl;

	std::cout << "Validation: " << privateKey.Validate( random, 3 ) << std::endl;
	return 0;
}

Possible output:

value1: 63001.
value2: 17.
value3: 103.
value4: 251.
value5: 251.
value6: 103.
value7: 103.
value8: 0.
Assertion failed: rsa.cpp(270): CryptoPP::InvertibleRSAFunction::Validate
Assertion failed: rsa.cpp(274): CryptoPP::InvertibleRSAFunction::Validate
Assertion failed: rsa.cpp(276): CryptoPP::InvertibleRSAFunction::Validate
Assertion failed: rsa.cpp(278): CryptoPP::InvertibleRSAFunction::Validate
Assertion failed: rsa.cpp(280): CryptoPP::InvertibleRSAFunction::Validate
Assertion failed: rsa.cpp(285): CryptoPP::InvertibleRSAFunction::Validate
Validation: 0
Press any key to continue . . .

Tests are passed:

.\cryptest.exe v
All tests passed!

Seed used was 1659696024
Test started at Fri Aug 5 13:40:24 2022
Test ended at Fri Aug 5 13:43:21 2022

and

.\cryptest.exe tv all
Tests complete. Total tests = 17549. Failed tests = 0.