opensearch-api-specification icon indicating copy to clipboard operation
opensearch-api-specification copied to clipboard

Published 20 hours ago • verified publisher icon opensearch-project

[PROPOSAL] Add integration tests

Open dblock opened this issue 9 months ago • 0 comments

What/Why

What are you proposing?

Coming from https://github.com/opensearch-project/opensearch-api-specification/pull/271#issuecomment-2088669972, we don't actually know whether the specs authored here are correct. Add integration tests that exercise these specs.

What users have asked for this feature?

See https://github.com/opensearch-project/opensearch-api-specification/pull/271#issuecomment-2088669972.

What problems are you trying to solve?

  • Test that the specs authored match what OpenSearch code is doing.
  • Early signal if OpenSearch is breaking specs.

What is the developer experience going to be?

Every PR will start various versions of OpenSearch, load the specs, run tests against the local version of OpenSearch.

What will it take to execute?

Unsure. Maybe there's an existing tool? Otherwise I would start with something simple that executes all GET requests and compares outputs against a previous recording. Then author requests/responses and compare. Finally, we need coverage to know how many of the APIs are invoked, how many request/response bodies are matched.

dblock avatar May 01 '24 16:05 dblock