Enhancement: floating point capture and tolerance.

[rainwoodman]

We'd like to use FileCheck to check log files with floating point numbers (in scientific computing). Apparently neither the C++ filecheck nor the FileCheck.py knows about floating point numbers.

Here I propose a capture based syntax for checking if floating points numbers are within a tolerance range:

inlined tolerance check:

Pass:

# CHECK: A + B = [[.*:VAR:tolerance(1.0,0,0.1)]]
echo "A + B = 1.0 extra text"

Fail:

# CHECK: A + B = [[.*:VAR:tolerance(1.0,0,0.1)]]
echo "A + B = 1.2 extra text"

The capture expression is extended with an extra ":" segment, where if the segment is

• tolerance(val, atol, rtol) then the captured string is converted to floating point and checked against val with the given tolerance Refer to numpy.allclose().

full tolerance check:

A new CHECK-TOLERANCE check for captured variables, The right hand side of = can be several tolerance expressions joined by or.

# CHECK: A + B = [[.*:VAR]]
# CHECK-TOLERANCE: [[VAR]] = tolerance(1.0, 0, 0.1) or tolerance(2.0, 0, 0.1)
echo "A + B = 1.0"
# CHECK: A + B = [[.*:VAR]]
# CHECK-TOLERANCE: [[VAR]] = tolerance(1.0, 0, 0.1) or tolerance(2.0, 0, 0.1)
echo "A + B = 2.0"

Implementation: I think it is relatively easy to add both or either form of floating point checks to FileCheck.py, but doing so will break the compatibility with LLVM filecheck. What is FileCheck.py's policy on this?

[stanislaw]

The policy is very simple: there are very minor differences so far but the Python version is definitely ok to diverge further as long as the documentation clearly explains it. I was recently thinking of adding a CHECK-ONCE option, for example, so we definitely are not bound to the LLVM version.

I like both of your options but if I had to choose one, I would probably implement your full version. But having both implemented is certainly great to have too!