opensearch-api-specification icon indicating copy to clipboard operation
opensearch-api-specification copied to clipboard

[BUG] indices/segments.yaml is flaky

Open karenyrx opened this issue 8 months ago • 0 comments

What is the bug?

tests/default/indices/segments.yaml test is flaky.

Runs 2 and 3 of this CI job failed, but 4th try succeded without any code changes. https://github.com/opensearch-project/opensearch-api-specification/actions/runs/14722249535/job/41388391724


FAILED  indices/segments.yaml (tests/default/indices/segments.yaml)
    FAILED  CHAPTERS
        FAILED  Get details about Lucene segments.
            PASSED  PARAMETERS
                PASSED  human
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            FAILED  RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_0/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_1/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/, ...)
            SKIPPED RESPONSE OUTPUT VALUES
        FAILED  Get details about Lucene segments with human fields.
            PASSED  PARAMETERS
                PASSED  human
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            FAILED  RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_0/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/shards/0/0/segments/_1/sort property is not defined in the spec, data/indices/top_queries-2025.04.30-51514/, ...)
            SKIPPED RESPONSE OUTPUT VALUES
        PASSED  Get details about Lucene segments inside the specified index.
            PASSED  PARAMETERS
                PASSED  index
                PASSED  expand_wildcards
                PASSED  allow_no_indices
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            PASSED  RESPONSE PAYLOAD SCHEMA
            SKIPPED RESPONSE OUTPUT VALUES

How can one reproduce the bug?

Reproducible locally at times with latest main as well:

~/opensearch-api-specification on [main] % OPENSEARCH_PASSWORD=myStrongPassword123! npm run test:spec -- --opensearch-insecure --tests=tests/default/indices/segments.yaml

> [email protected] test:spec
> ts-node tools/src/tester/test.ts --opensearch-insecure --tests=tests/default/indices/segments.yaml

OpenSearch 2.19.1


FAILED  segments.yaml (tests/default/indices/segments.yaml)
    FAILED  CHAPTERS
        FAILED  Get details about Lucene segments.
            PASSED  PARAMETERS
                PASSED  human
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            FAILED  RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.29-51492/shards/0/0/segments/_5/sort property is not defined in the spec, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/to, ...)
            SKIPPED RESPONSE OUTPUT VALUES
        FAILED  Get details about Lucene segments with human fields.
            PASSED  PARAMETERS
                PASSED  human
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            FAILED  RESPONSE PAYLOAD SCHEMA (data/indices/top_queries-2025.04.29-51492/shards/0/0/segments/_5/sort property is not defined in the spec, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/top_queries-2025.04.29-51492/shards/0 must be object, data/indices/to, ...)
            SKIPPED RESPONSE OUTPUT VALUES
        PASSED  Get details about Lucene segments inside the specified index.
            PASSED  PARAMETERS
                PASSED  index
                PASSED  expand_wildcards
                PASSED  allow_no_indices
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            PASSED  RESPONSE PAYLOAD SCHEMA
            SKIPPED RESPONSE OUTPUT VALUES


Tested 2/599 paths.

Commits on main, for reproducing the error, at time of filing this issue:

~/opensearch-api-specification on [main] % git log --oneline
f82f42d (HEAD -> main, upstream/main) [Docs]: Add and edit Node API descriptions (#872)
5fe396d Fix bad link (#877)
b06e2f4 Remove unnecessary string types (#874)

What is the expected behavior?

Tests to be reliable.

What is your host/environment?

GHA

Do you have any screenshots?

Image

Do you have any additional context?

karenyrx avatar May 03 '25 18:05 karenyrx