Mbed-TLS
Extend typo check to test data/function files?
We sometimes make typos in dependencies in test data/function files, resulting in a test that will be skipped in all builds. For example this happened here.
This could probably be caught by the typo checker in tests/scripts/check_names.py.
Alternatively, there's a long-standing idea from @gilles-peskine-arm to use to a analysis step at the end of CI runs checking if there are test cases that were skipped in all configurations, which would catch this too. However, extending the typo checker seems easier and could be an intermediate step while waiting for us to implement post-run analysis.
there's a long-standing idea from @gilles-peskine-arm to use to a analysis step at the end of CI runs checking if there are test cases that were skipped in all configurations
This is already implemented. See the logs from a “Result analysis” step of a CI run. All that's missing to have the CI fail if a test case is never executed is changing one occurrence of warning into error. The remaining work is to analyze the current report, implement an allowlist for the legitimate cases and fix the non-legitimate cases.
The typo check would still be nice to have in .function files, for code. For example, right now in development, we have a line
#if !defined(MBEDTLS_USE_PSA_CRYPO) || !defined(MBEDTLS_TEST_DEPRECATED)
I'd like to make this more constraining: names defined in test code should not be used in the library. And, on a very much related note, names defined in library/* should not be used in include/*. So I've filed a more general task to support these separate domains in check_names.