[BUG] The test ensemble_propagate fails on FreeBSD

[yurivict]

$ ./work/.build/test/ensemble_propagate
[2022-01-27 10:50:51.631] [heyoka] [info] heyoka logger initialised

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ensemble_propagate is a Catch v2.12.1 host application.
Run with -? for options

-------------------------------------------------------------------------------
scalar propagate until
-------------------------------------------------------------------------------
/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:47
...............................................................................

/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:95: FAILED:
  REQUIRE( std::get<0>(res[i]).get_state() == ta.get_state() )
with expansion:
  { 0.912945, 0.408082 }
  ==
  { 0.912945, 0.408082 }

-------------------------------------------------------------------------------
scalar propagate for
-------------------------------------------------------------------------------
/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:138
...............................................................................

/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:187: FAILED:
  REQUIRE( std::get<0>(res[i]).get_state() == ta.get_state() )
with expansion:
  { 0.912945, 0.408082 }
  ==
  { 0.912945, 0.408082 }

-------------------------------------------------------------------------------
scalar propagate grid
-------------------------------------------------------------------------------
/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:231
...............................................................................

/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:285: FAILED:
  REQUIRE( std::get<0>(res[i]).get_state() == ta.get_state() )
with expansion:
  { 0.912945, 0.408082 }
  ==
  { 0.912945, 0.408082 }

-------------------------------------------------------------------------------
batch propagate until
-------------------------------------------------------------------------------
/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:297
...............................................................................

/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:355: FAILED:
  REQUIRE( std::get<0>(res[i]).get_state() == ta.get_state() )
with expansion:
  { 0.912945, 0.912945, 0.408082, 0.408082 }
  ==
  { 0.912945, 0.912945, 0.408082, 0.408082 }

-------------------------------------------------------------------------------
batch propagate for
-------------------------------------------------------------------------------
/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:396
...............................................................................

/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:456: FAILED:
  REQUIRE( std::get<0>(res[i]).get_state() == ta.get_state() )
with expansion:
  { 0.912945, 0.912945, 0.408082, 0.408082 }
  ==
  { 0.912945, 0.912945, 0.408082, 0.408082 }

-------------------------------------------------------------------------------
batch propagate grid
-------------------------------------------------------------------------------
/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:499
...............................................................................

/disk-samsung/freebsd-ports/math/heyoka/work/heyoka-0.17.0/test/ensemble_propagate.cpp:568: FAILED:
  REQUIRE( std::get<0>(res[i]).get_state() == ta.get_state() )
with expansion:
  { 0.912945, 0.912945, 0.408082, 0.408082 }
  ==
  { 0.912945, 0.912945, 0.408082, 0.408082 }

===============================================================================
test cases:    6 |    0 passed | 6 failed
assertions: 7403 | 7397 passed | 6 failed

Cleaning up global MPFR caches.

[bluescarni]

I am totally stumped on these failures!

In this case, there should be no tolerances involved - the tests are checking that multiple separate numerical integrations taken in parallel via multithreading produce identical results to the integrations run sequentially one after the other.

The numbers seem to match up to the printed digits, so the differences seem to arise in the last few digits.

It would be useful to know:

1. for which floating-point type are these test failing (i.e., double, long double, ...)
2. the exact difference between produced vs expected values.

For point 1), could you try to redefine the tuple at line 34 to contain only a single floating-point type, e.g., instead of

const auto fp_types = std::tuple<double
#if !defined(HEYOKA_ARCH_PPC)
                                 ,
                                 long double
#endif
#if defined(HEYOKA_HAVE_REAL128)
                                 ,
                                 mppp::real128
#endif
                                 >{};

just

const auto fp_types = std::tuple<double>{};

and

const auto fp_types = std::tuple<long double>{};

?

For point 2), could you try to to add printing statements right before line 95, something like:

std::cout.precision(std::numeric_limits<fp_t>::max_digits10);
std::cout << std::get<0>(res[i]).get_state()[0] - ta.get_state()[0] << '\n';

and report the output right before the failure?

[bluescarni]

Also, are these Debug or Release builds?

[bluescarni]

@yurivict any chance you could check this out as well?

[yurivict]

With 2 changes that you suggested this case passes.

It prints a lot of zeros (lines with one zero in each) before it says that is passes.

[bluescarni]

Thanks for checking!

Do the tests pass with only double, long double or both?

[bluescarni]

@yurivict I have a PR up at #254 which disables long double tests on FreeBSD. Would you mind give it a spin?

I'd like to make a new release soon, so it would be great if we could close this report.

[bluescarni]

@yurivict I will go ahead and close this, please feel free to re-open if needed.