[Spaces] Add Scout tests using Saved Objects API `_import` endpoint in different spaces
This PR adds new Scout tests using the Saved Objects _import endpoint in different spaces (default space and custom spaces).
While the initial goal was to migrate all FTR tests belonging to the x-pack/platform/test/saved_object_api_integration/spaces_only/config.ts test config to Scout, it turned out that:
- Several Saved Object APIs operations are deprecated, except for some (e.g.
_importand_export). I didn't deem worth our time to move - The FTR test suite is quite involved and it is importing an archive with outdated saved objects types. A 1:1 migration would involve rearchitecting test suite.
With this said, and despite the limited coverage, I think it's still worth to add these Scout tests increase the number of Scout tests in the Kibana Security area. This will let us additional data points to see how Scout performs with custom spaces. This PR is no longer a "migration" because no FTR test is deleted.
@csr Thank you for putting this together! I have some questions regarding the scope and how we should plan to continue the effort of migrating tests to scout.
This suite focuses purely on space isolation and namespace behavior, testing how objects are created, retrieved, updated, and deleted across different Kibana spaces. ... most Saved Object API operations are deprecated so they aren't used in the new Scout tests, except for the bulk delete operation. The import and export APIs aren't deprecated so these tests focus especially on those.
I noticed that this PR covers testing the import API exclusively, and the description might just be out of date.
- Can we update the title and description of the PR for the sake of posterity?
- Are we limited to testing only public APIs with Scout?
- How would you recommend we move forward with Scout test coverage knowing that the public APIs are deprecated, but the operations of the SO repository (SOR) are not? It will be important for us to keep or improve the coverage we have now with the pre-scout FTR to ensure that any changes made in the SOR do not introduce regressions. Maybe a good topic for our next cross-team sync.
Hey @jeramysoucy, thank you very much for your feedback! We've synced offline, but I also wanted to follow up here with our discussion:
Can we update the title and description of the PR for the sake of posterity?
Good point, I have updated both the title and description. Feel free to suggest further improvements 👍
Are we limited to testing only public APIs with Scout?
We're not limited to testing public APIs with Scout. We encourage teams to test both public and private APIs.
How would you recommend we move forward with Scout test coverage knowing that the public APIs are deprecated, but the operations of the SO repository (SOR) are not?
I agree we should maintain the FTR test suite to not lose coverage 💯
I assume Saved Object Operations are accessible via the kbnClient (correct me if I'm wrong). In that case, Scout API tests aren't likely the way to go to test them as they shouldn't rely on kbnClient to test the final endpoint. Perhaps the solution here is to test Saved Object Operations using Jest unit integration tests (those spin up clusters).
:yellow_heart: Build succeeded, but was flaky
- Buildkite Build
- Commit: 25e8f6e87327cb8c5e548461686815242172e761
Failed CI Steps
Test Failures
- [job] [logs] Jest Tests #1 / GroupedAlertsTable resets all levels pagination when global query updates
- [job] [logs] Jest Tests #1 / GroupedAlertsTable resets all levels pagination when selected group changes
- [job] [logs] Jest Tests #1 / GroupedAlertsTable resets innermost level's current page when that level's page size updates
- [job] [logs] Jest Tests #1 / GroupedAlertsTable resets only most inner group pagination when its parent groups open/close
- [job] [logs] Jest Tests #1 / GroupedAlertsTable resets outermost level's current page when that level's page size updates
- [job] [logs] Jest Tests #1 / useQueryAlerts fetch alerts data
- [job] [logs] Jest Tests #1 / xy style settings Axis settings should pass in endzone visibility setter and current sate for time chart
- [job] [logs] Jest Tests #1 / xy style settings Axis settings should pass in extent information
Metrics [docs]
Public APIs missing comments
Total count of every public API that lacks a comment. Target amount is 0. Run
node scripts/build_api_docs --plugin [yourplugin] --stats commentsfor more detailed information.
| id | before | after | diff |
|---|---|---|---|
@kbn/scout |
216 | 221 | +5 |
History
- :broken_heart: Build #371306 failed 5c7270d816faf616a8c16e3b7082ac6c01f18cbf
- :broken_heart: Build #371169 failed 050908b4e29682681c2df96d43d8992b917d27c9
- :broken_heart: Build #367870 failed 2d47a96258aeea5db27e35d8988ac9454d3b0503
- :broken_heart: Build #367820 failed 857acd13429dd89db4beb0d947ee30394152ce99
- :broken_heart: Build #367039 failed 960498156a8d7205f1c2b8cbfab6d5948302e63c
cc @csr
Starting backport for target branches: 8.19, 9.1, 9.2
https://github.com/elastic/kibana/actions/runs/20139115681
💔 All backports failed
| Status | Branch | Result |
|---|---|---|
| ❌ | 8.19 | Backport failed because of merge conflicts |
| ❌ | 9.1 | Backport failed because of merge conflicts |
| ❌ | 9.2 | Backport failed because of merge conflicts |
Manual backport
To create the backport manually run:
node scripts/backport --pr 244323
Questions ?
Please refer to the Backport tool documentation
Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 244323 locally
cc: @csr
💚 All backports created successfully
| Status | Branch | Result |
|---|---|---|
| ✅ | 9.2 | |
| ✅ | 9.1 | |
| ✅ | 8.19 |
Note: Successful backport PRs will be merged automatically after passing CI.
Questions ?
Please refer to the Backport tool documentation
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. cc: @csr
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. cc: @csr
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. cc: @csr
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. cc: @csr
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. cc: @csr