opensearch-api-specification
opensearch-api-specification copied to clipboard
[FEATURE] Add support for pending tests
Is your feature request related to a problem?
In writing tests for _reindex
I tried the following.
- synopsis: Reindex from a remote cluster.
path: /_reindex
method: POST
request_body:
payload:
source:
remote:
host: https://localhost:9200
username: admin
password: password
socket_timeout: 30s
connect_timeout: 30s
index: movies
dest:
index: films
response:
status: 200
payload:
total: 1
This doesn't work because it needs a setting and an SSL cert.
I'd like to include the test, but mark it as pending, not run it, and show the explanation of why it wasn't run.
What solution would you like?
We had https://github.com/opensearch-project/opensearch-api-specification/pull/358 that would skip tests, but I'd actually like the test to run, but not fail.
In the test I want to write this:
- synopsis: Reindex from a remote cluster.
pending: |
SSL needs to be configured to trust itself.
For this to work, will need to:
1. set reindex.remote.allowlist=localhost:9200 in docker-compose.yml
2. setup SSL in a way that the docker instance trusts its own cert
path: /_reindex
method: POST
Then at runtime it should say
IGNORED (error, SSL needs to be configured to trust itself.)
If the test were to pass, it would actually fail because it shouldn't have been marked pending.
What alternatives have you considered?
- Comment out tests.
- https://github.com/opensearch-project/opensearch-api-specification/pull/358