opensearch-api-specification
opensearch-api-specification copied to clipboard
[BUG] indices/segments.yaml is flaky
What is the bug?
tests/default/indices/segments.yaml test is flaky.
Runs 2 and 3 of this CI job failed, but 4th try succeded without any code changes. https://github.com/opensearch-project/opensearch-api-specification/actions/runs/14722249535/job/41388391724
FAILED indices/segments.yaml (tests/default/indices/segments.yaml)
FAILED CHAPTERS
FAILED Get details about Lucene segments.
PASSED PARAMETERS
PASSED human
PASSED REQUEST BODY
PASSED RESPONSE STATUS
PASSED RESPONSE PAYLOAD BODY
FAILED RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_0/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_1/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/, ...)
SKIPPED RESPONSE OUTPUT VALUES
FAILED Get details about Lucene segments with human fields.
PASSED PARAMETERS
PASSED human
PASSED REQUEST BODY
PASSED RESPONSE STATUS
PASSED RESPONSE PAYLOAD BODY
FAILED RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_0/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_1/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/, ...)
SKIPPED RESPONSE OUTPUT VALUES
PASSED Get details about Lucene segments inside the specified index.
PASSED PARAMETERS
PASSED index
PASSED expand_wildcards
PASSED allow_no_indices
PASSED REQUEST BODY
PASSED RESPONSE STATUS
PASSED RESPONSE PAYLOAD BODY
PASSED RESPONSE PAYLOAD SCHEMA
SKIPPED RESPONSE OUTPUT VALUES
How can one reproduce the bug?
Reproducible locally at times with latest main as well:
~/opensearch-api-specification on [main] % OPENSEARCH_PASSWORD=myStrongPassword123! npm run test:spec -- --opensearch-insecure --tests=tests/default/indices/segments.yaml
> [email protected] test:spec
> ts-node tools/src/tester/test.ts --opensearch-insecure --tests=tests/default/indices/segments.yaml
OpenSearch 2.19.1
FAILED segments.yaml (tests/default/indices/segments.yaml)
FAILED CHAPTERS
FAILED Get details about Lucene segments.
PASSED PARAMETERS
PASSED human
PASSED REQUEST BODY
PASSED RESPONSE STATUS
PASSED RESPONSE PAYLOAD BODY
FAILED RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.29-51492/shards/0/0/segments/_5/sort property is not defined in the spec, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/to, ...)
SKIPPED RESPONSE OUTPUT VALUES
FAILED Get details about Lucene segments with human fields.
PASSED PARAMETERS
PASSED human
PASSED REQUEST BODY
PASSED RESPONSE STATUS
PASSED RESPONSE PAYLOAD BODY
FAILED RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.29-51492/shards/0/0/segments/_5/sort property is not defined in the spec, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/to, ...)
SKIPPED RESPONSE OUTPUT VALUES
PASSED Get details about Lucene segments inside the specified index.
PASSED PARAMETERS
PASSED index
PASSED expand_wildcards
PASSED allow_no_indices
PASSED REQUEST BODY
PASSED RESPONSE STATUS
PASSED RESPONSE PAYLOAD BODY
PASSED RESPONSE PAYLOAD SCHEMA
SKIPPED RESPONSE OUTPUT VALUES
Tested 2/599 paths.
Commits on main, for reproducing the error, at time of filing this issue:
~/opensearch-api-specification on [main] % git log --oneline
f82f42d (HEAD -> main, upstream/main) [Docs]: Add and edit Node API descriptions (#872)
5fe396d Fix bad link (#877)
b06e2f4 Remove unnecessary string types (#874)
What is the expected behavior?
Tests to be reliable.
What is your host/environment?
GHA