open-metadata
The value in "Total tests" does not match the actual number of executed tests.
Description: In OpenMetadata, I create a test suite, test case, and add the ready-made test results via API. These are associated with a custom testDefinition. All tests are non-empty, and reindexing completes successfully (with the 'Recreate Indexes' option). However, for some tables, we observe an incorrect display of 'total tests'—it is lower than the 'test cases' value, even though all test cases are non-empty.
We are currently using OpenMetadata version 1.7.5, but I checked on version 1.9.7, and the issue persists. This primarily occurs with tables that have a large number of columns. For example, take the act_hi_taskinst table in openmetadata_db, which has 36 columns. When creating tests for various columns, at some point, 'total tests' stops reflecting the actual count. In my case, creating tests for 16 different columns was enough to reproduce the issue. As I understand it, the 'total tests' value is derived from the 'Get the execution summary of test suites' query. Let me describe the situation where not all column tests appear in the 'execution summary': If I delete test_case_1 for column_1, which is included in the 'execution summary', the 'total tests' value remains unchanged. However, the 'execution summary' then includes information about testing column_2 (with test_case_2), which was initially missing. When I restore test_case_1, the information about column_1 reappears in 'total tests', while the information about column_2 and test_case_2 disappears.
I have attached a screenshot of the error for version 1.9.7.
Expected behavior: For any table, the 'total tests' value shows the count of all non-empty test cases.
Environment: • OpenMetadata version: 1.7.5, 1.9.7 • Browser: Chrome • OS: Linux (Server)
@TeddyCr No, we don’t have any tests that have been created but never executed. In OpenMetadata, we add pre-executed test results via the API, and all test cases are non-empty.
@MASemenova are you still facing this issue in the latest stable release? Closing it, if you are still seeing the issue please reopen it.