[Lens] Move dataViews cache into main Lens state

[dej611]

Summary

This PR is a required step for #129299 and more future features in Lens. The dataViews state was historically stored into the specific indexPatterns datasource, which was one of many potential datasources of the editor, but since Visualization starts to know more about it, it had to be lifted up to the editor main state to be reachable.

A new dataViews lens inner group has been created in the main Lens state, which holds refs, dataViews cache, fields existence and potential errors states. Also the loading phase has been refactored to inspect directly the SO references to load only required dataViews (this sounds like a possible assumption as there is no hidden dataViews concept in Lens yet).

All the dataViews management method, which used to be stored into datasource are now stored within the indexPatternService, which is a tiny API around few functions that can be shared across the Lens app hierarchy. The current state has not been stored into the indexPatternService as this is used also outside of the frame and it was easier to manage two variables rather than duplicate all the code for the TopNav + EditorFrame + Embeddable .

• [x] Move the dataViews state into the main root
• [x] IndexPatternServiceAPI
• [x] Make sure Embeddables work fine with that
  • [x] Make it work in Dashboard
  • [x] Examples
• [x] Open Saved Lens visualization
  • [x] From dashboard
  • [x] To Discover
• [x] Make sure Dataviews change work
  • [x] Top Nav
  • [x] Layer
• [x] Update persistable serializator
• [ ] Migrations
• [x] Fix types
• [ ] Tests
  • [ ] Broken tests
  • [x] Checks all visualizations

Checklist

Delete any items that are not applicable to this PR.

• [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• [ ] Documentation was added for features that require explanation or tutorials
• [ ] Unit or functional tests were updated or added to match the most common scenarios
• [ ] Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• [ ] This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• [ ] This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

• [ ] This was checked for breaking API changes and was labeled appropriately

[dej611]

@mbondyra the layer dataViews switch bug should have been addressed in the latest version

[elasticmachine]

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

[flash1293]

Conflicts are due to the improved error handling, shouldn't be too difficult to resolve

[dej611]

Conflicts should be solved now.

[kibana-ci]

:green_heart: Build Succeeded

• Buildkite Build
• Commit: 958ae77cb6652804d891087e9d734066ba3ad2e7

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
lens	903	906	+3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
lens	530	535	+5

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
lens	1.2MB	1.2MB	+7.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
lens	34.6KB	34.6KB	+60.0B

Unknown metric groups

API count

id	before	after	diff
lens	614	621	+7

History

• :green_heart: Build #64729 succeeded 1b64dda6316320f9d68869e5aca9430944284c33
• :green_heart: Build #64556 succeeded d3175135848b078596de1b95570683d1647e2314
• :green_heart: Build #64528 succeeded 07e3136c5372ae3d4f1dff9dbeda4ddb3fd7d944
• :broken_heart: Build #64479 failed 5a5d3b9111a44a6fa228e15864187a7372b7ae10
• :broken_heart: Build #64455 failed d32f9dd41ea941506e2ff06e3d2323f8ef298907
• :broken_heart: Build #64264 failed d594e09465376f4e808cf9fd757e938bf258d17c

To update your PR or re-run it, just comment with: @elasticmachine merge upstream

[mbondyra]

Retested and couldn't find anything not working! Amazing work!