opensearch-benchmark
opensearch-benchmark copied to clipboard
opensearch-project
[Testing Confirmation] Confirm current testing requirements
As part of the discussion around implementing an organization-wide testing policy, I am visiting each repo to see what tests they currently perform. I am conducting this work on GitHub so that it is easy to reference.
Looking at the OpenSearch Benchmark repository, it appears there is
| Repository | Unit Tests | Integration Tests | Backwards Compatibility Tests | Additional Tests | Link |
|---|---|---|---|---|---|
| OpenSearch Benchmark | https://github.com/opensearch-project/opensearch-benchmark/issues/222 |
I don't see any requirements for code coverage in the testing documentation. If there are any specific requirements could you respond to this issue to let me know?
If there are any tests I missed or anything you think all repositories in OpenSearch should have for testing please respond to this issue with details.
We are currently working on updating OpenSearch Benchmark to work with the latest Python version and getting the release out. Once that is completed, we will revisit the code coverage requirements and follow up in this issue.
Hi @gkamat, thank you for following up. I will make sure to note your feedback.
Hi @scrawfor99, are you requesting that we just confirm if the above is accurate or if we should also include code-coverage? If the latter, do you have any SOP on how we can onboard code coverage to OSB?
Hi @IanHoang, I am just trying to confirm that the above is accurate. There is no specific guidelines I am looking for. The plan is to collect info from each repo and then come up with an organizational policy so we can all meet an established threshold. There is no "right" or "wrong" thing to be doing at this point.
@scrawfor99 Sorry for the late response, but yes the above is accurate. Currently OSB just uses unit tests and integration testing. @gkamat is currently diving into integration tests to determine why they're flaky. As @gkamat mentioned earlier, we plan to add code coverage in the future.