Build problems due to gmpy on Debian bookworm, Fedora 38

[alext-w]

Fisrt of all, thank you for releasing this library, I think it's quite valuable for the eternal quest of better-quality crypto implementations.

The library install/build process fails on newer distros (detailed error example at the bottom of this post), mostly because of gmpy, so I wonder why that long-unsupported binding is used, and not gmpy2 - could you please clarify that choice?

It wasn't a problem as of Debian bullseye ("latest" at the time the library was released, I reckon), but now, for example, the current Dockerfile won't build because the "latest" is now bookworm, and there, due to some header-related inconsistency between CPython and Cython for Python 3.11 all these distros have, gmpy build no longer finds one of the headers and bails out.

For my experiments, I've ported the paranoid lib to use gmpy2 to get it building on those newer distros, and have almost all self-tests passing, except the GMP RNG-focused batch, which fails because gmpy2 uses the default GMP RNG, which is MT these days (and for a while), and not LCG the tests assume, for those I deferred until having discussed this with you. The underlying library functionality seems to work fine after porting, it's just the test set that needs corrections.

Would you be interested in getting this patch (or fixing that yourselves, the gmpy->gmpy2 "porting" is quite simple and mostly mechanical)?

gmpy build error example

27.99   
27.99   × Running setup.py install for gmpy did not run successfully.
27.99   │ exit code: 1
27.99   ╰─> [7 lines of output]
27.99       /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
27.99         warnings.warn(
27.99       src/gmpy.c:243:10: fatal error: longintrepr.h: No such file or directory
27.99         243 | #include "longintrepr.h"
27.99             |          ^~~~~~~~~~~~~~~
27.99       compilation terminated.
27.99       error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
27.99       [end of output]
27.99   
27.99   note: This error originates from a subprocess, and is likely not a problem with pip.
27.99 error: legacy-install-failure
27.99 
27.99 × Encountered error while trying to install package.
27.99 ╰─> gmpy

[alext-w]

Let me try pinging the last two committers I see in the history: @bleichen, @pedroysb, could you please comment on the above? Is the project alive at all, just to make sure?

[bleichenbacher-daniel]

I believe the only reason to use gmpy instead of gmpy2 was better support of gmpy within Google at the time the project started. I'm no longer at Google and hence can't comment on whether the situation has changed. Otherwise gmpy2 has better integer arithmetic than gmpy and would indeed be preferable.

The failures of the unit tests for the gmpy random number generators only mean that all the claims for this random number generator are no longer valid (rsp. no longer reproducible).

The only reason to include the old gmpy random number generators is that the provided a simple way to generate weak input data for the randomness tests. I.e., we use a large number of weak random number generators and check if our tests can detect them. If detection is poor we tried to find and add additional checks. One observation was that the spectral test included in the NIST test suite did a relatively poor job at detecting LCGs. LLL-based methods are far more powerful and can detect LCGs even if the weak randomness is hidden within ECDSA signatures.

[alext-w]

Thanks Daniel! Do you have any plans for developing this project further, if Google wouldn't be interested in doing that? As far as I can see in your recent comments in the Wycheproof repo, you do plan to do that for Wycheproof, at least as much as your free time allows for it (right?), so I wonder if it's the same for this one.

[bleichenbacher-daniel]

Pedro would have to tell us what plans exist within Google. The public key validation depends a lot on having sufficient resources to test large sets of public keys. While I have some small new ideas I can't do much there.

The randomness checks are an extension of NIST SP 800-22. NIST is currently revising this document. Once a draft of the new version is available, I'm certainly interested to review this and check what can be done. There are also some open questions that may lead to a paper.

[alext-w]

@pedroysb, any comment you could provide here, please?

[pedroysb]

Hi @alext-w , thanks for your interest in collaborating. Google has support to gmpy2 now so your patch is welcome.

This project isn't an official Google product and I'm the only one still working on it. But we continue having sufficient resources to run our tests.

[alext-w]

Thanks Pedro, sounds good! Let me work out the formalities for the PR at my end and submit it, so that we could discuss this further (e.g., what to do with those LCG-assuming randomness tests after moving to gmpy2).

[pedroysb]

Hi @alext-w , any update regarding your PR? I'm looking forward to see it.

[alext-w]

Thanks for following up! Those formalities I mentioned are almost done by now (just a couple of last steps in progress), so my current plan is to submit a PR sometime this week. Almost there :)

[alext-w]

One of those last steps turned out to be a whole set of activities in and of itself (related to the CLA), so it'll take some more time for me to go through that, JFYI.

[alext-w]

Just wanted to check in - this is still in progress, but looks like I can see the end of it. Apologies it's taking so long!

[alext-w]

The PR is in. Thanks for your patience, it took a while, but now that I've walked the full process from my side once and got added to the CLA, the subsequent contributions should take less time.