kibana
kibana copied to clipboard
elastic
[ES|QL] Integration tests
Summary
The validation test suite now dumps all test cases into a JSON with a flag that tell if the query should fail or not.
On top of that an integration test has been written, who collects the JSON and execute it again against the ES ES|QL _query endpoint, then compare the two result.
- if both results are the same (both pass or errors) then it's good to go ✅
- if client side fails and ES pass, then it's a failure ❌
- if client side passes and ES fails, then it's bad but still ok ⚠️
- in this case the error is logged and stored into a JSON (not committed)
All tests are ran 8 times ( stringField => ["text", "keyword"], numberField => ["integer", "double", "long", "unsigned_long] for a totale of ~16k assertions (2000 x 8 tests).
Assertions have to be within a single test to avoid memory issues on the CI otherwise.
Fixes the signature utility for functions with a minimum number of args.
Checklist
- [x] Unit or functional tests were updated or added to match the most common scenarios
@dej611 I understand that you added the fix label due to the signature fix. The problem is that the docs team extracts the release notes from the title so the title here is not helping to understand what is being fixed.
- either change the title to indicate the fix
- or add the skip label as the bug fix is not so important and we can skip the mention
@dej611 I understand that you added the fix label due to the signature fix. The problem is that the docs team extracts the release notes from the title so the title here is not helping to understand what is being fixed.
- either change the title to indicate the fix
- or add the skip label as the bug fix is not so important and we can skip the mention
While doing the integration tests I found out some bugs in the autocomplete which I fixed here. In retrospect I should have left the integration tests and fix the bugs in a separate PR. Will change the label and extract the fixes somewhere else.
:green_heart: Build Succeeded
- Buildkite Build
- Commit: cb3074577cf7f2992b09b7a5aa16c6993e9a117e
Metrics [docs]
✅ unchanged
History
- :green_heart: Build #194064 succeeded abb9ecab7d75582cd9312857e991b27eb39c414a
- :yellow_heart: Build #194005 was flaky 150ffc79f1663ee251c139a053a5856ae51ead01
- :green_heart: Build #193882 succeeded 6356a460be248338e8ce2e246f7061eaf1e9360d
- :broken_heart: Build #193828 failed b27dc66cdfe6cb83bbe4e1938e14e88f54c581a5
To update your PR or re-run it, just comment with:
@elasticmachine merge upstream
This PR now contains only the integration tests. All the bug fix logic has been exported in https://github.com/elastic/kibana/pull/177165