iree-e2e-matmul-test duplicate flags

[Groverkss]

What happened?

On running:

<BUILD_DIR>/tools/testing/e2e/iree-e2e-matmul-test --help

there are duplicate flag options for test_utils.h:

# ===----------------------------------------------------------------------===
# Flags in iree/tools/testing/e2e/test_utils.h
# ===----------------------------------------------------------------------===

# Requires floating point result elements to match exactly.
--require_exact_results=true

# Requires floating point result elements to match exactly.
--require_exact_results=true

# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05

# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05

# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000

# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000

On changing the flags, --help only changes one of the flags:

iree-e2e-matmul-test --require_exact_results=false --help

# ===----------------------------------------------------------------------===
# Flags in iree/tools/testing/e2e/test_utils.h
# ===----------------------------------------------------------------------===

# Requires floating point result elements to match exactly.
--require_exact_results=false

# Requires floating point result elements to match exactly.
--require_exact_results=true

# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05

# Maximum absolute difference allowed with inexact floating point results.
--acceptable_fp_delta=1e-05

# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000

# Maximum number of tensor elements to check for the given test. For larger buffers, only every n-th element will be checked for some n chosed to stay just under that threshold and to avoid being a divisor of the inner dimension size to avoid special patterns. As the check uses a slow reference implementation, this is a trade-off between test latency and coverage. The value 0 means check all elements.
--max_elements_to_check=10000

Steps to reproduce your issue

ninja iree-test-deps
<BUILD_DIR>/tools/testing/e2e/iree-e2e-matmul-test --help

What component(s) does this issue relate to?

No response

Version information

Commit Hash: c1cfbfc48f0abb07ff2cfd6335eeb541097acda2

Additional context

No response

[benvanik]

that test_utils.h is defining flags in the header - that's invalid - that means every file that includes the .h will define the flags. the header should expose some accessor methods and keep the flags in the test_utils.c file:

in test_utils.h:

bool iree_test_utils_require_exact_results(void);

in test_utils.c:

IREE_FLAG(...);
bool iree_test_utils_require_exact_results(void) { return FLAG_require_exact_results; }

[benvanik]

This was broken in #16849 /cc @pashu123

[pashu123]

This was broken in #16849 /cc @pashu123

I will add the fix. Thanks.

[pashu123]

This was broken in #16849 /cc @pashu123

I will add the fix. Thanks.

@benvanik PTAL https://github.com/iree-org/iree/pull/17299

[ScottTodd]

Looks like this was fixed by https://github.com/iree-org/iree/pull/17299