secp256k1 icon indicating copy to clipboard operation
secp256k1 copied to clipboard

Recalibrate unit tests

Open sipa opened this issue 2 years ago • 2 comments

The tests (src/tests.cpp and included modules) consist of some tests whose runtime is independent of the TEST_ITERS variable, and some which are. The ones that are independent take a significant amount of time (compared to the other ones, at a normal TEST_ITERS level). The ones that do depend on it do so in a rather inconsistent way.

So to improve upon that:

  • Turn constant-time tests into TEST_ITERS-proportional ones were possible.
  • Perhaps disable some very slow constant-time ones entirely at very low TEST_ITERS levels.
  • Run benchmarks on each of the proportional ones, and adjust to make sure they all get a somewhat representative amount of runtime in typical configurations.

sipa avatar Dec 07 '21 20:12 sipa

gprof of ./tests 1 shows that the new test_ecmult_constants() tests is indeed very heavy.

gprof-tests1.txt

real-or-random avatar Dec 09 '21 10:12 real-or-random

Related comment introduced in https://github.com/bitcoin-core/secp256k1/pull/1122/commits/17065f48ae261c6949dab74a7c197ac13b52eb1b:

https://github.com/bitcoin-core/secp256k1/blob/3efeb9da21368c02cad58435b2ccdf6eb4b359c3/src/tests.c#L7091-L7093

real-or-random avatar Jul 11 '22 12:07 real-or-random