fmpz_lll_is_reduced_d discrepancy with valgrind

[fredrik-johansson]

#include "fmpz_mat.h"
#include "fmpz_lll.h"

/*
matrix.txt:
4 6  0 0 -2 1 0 0 -67632831055744180472849 68480250255746521923500 -13517161229434763063995 -27034322458869525538406 109347158226885974831118 21180975364881058796086 55966194494895090742263 -56667434218646258429562 11185456272884349801226 22370912545768698648986 152655036552056581278552 39643837213073670887923 -8862599987648755993657 8973645722009393899953 -1771287569587258744802 -3542575139174517047296 -106422625925923694256093 255182450251415747447363
*/

int main()
{
    fmpz_mat_t M;
    fmpz_mat_init(M, 4, 6);

    fmpz_lll_t ctx;

    fmpz_lll_context_init(ctx, 0.75, 0.51, Z_BASIS, APPROX);

    FILE * fp = fopen("matrix.txt", "r");
    fmpz_mat_fread(fp, M);

    printf("\nis_reduced_d = %d\n", fmpz_lll_is_reduced_d(M, ctx));
}

This prints 0, 0 normally, but prints 1, 0 when run with valgrind. Indeed, with assertions enabled, there is an assertion error:

src/fmpz_lll/is_reduced_d.c:863: fmpz_lll_is_reduced_d: Assertion `(fl->rt == Z_BASIS ? fmpz_mat_is_reduced(B, fl->delta, fl->eta) : fmpz_mat_is_reduced_gram(B, fl->delta, fl->eta))' failed.

[fredrik-johansson]

Marking this as a bug, but it's not high priority to fix since so far it only manifests under valgrind.

[albinahlback]

To be clear, this only happens with --enable-assert?

[albinahlback]

I could reproduce, and I think it's a bug/feature in Valgrind.

Reading Valgrind's manual:

Rounding: Valgrind does observe the 4 IEEE-mandated rounding modes (to nearest, to +infinity, to -infinity, to zero) for the following conversions: float to integer, integer to float where there is a possibility of loss of precision, and float-to-float rounding. For all other FP operations, only the IEEE default mode (round to nearest) is supported.

Just woke up, but I read this as that all explicit rounding operation supports all IEEE modes. However, all other operations (including multiplication etc.) does not support any other mode other than the default one, and so we cannot expect fmpz_lll_is_reduced_d to work as intended under Valgrind.

[albinahlback]

I'm closing this. Please reopen if I misinterpreted it.