[RFC] What type of visualization abstractions do developers need?

[joshuarrrr]

Is your feature request related to a problem? Please describe.

OpenSearch Dashboards should provide users with a consistent, unified, and intuitive visualization experience throughout the application. To achieve that goal, we need better, easier-to-use visualization tools and interfaces for the developers building and creating those experiences.

Currently, it's difficult for developers to interact with and leverage the existing visualization plugins and frameworks in OpenSearch Dashboards, and the abstractions we do have don't clearly map to actual developer needs. Before we begin developing new abstractions, we first want to better understand, collect, and categorize all the existing visualization pain points and needs.

Describe the solution you'd like

Why invest in an abstraction layer?

1. Allows opinionated and consistent design decisions "baked-in"
2. Prevents dependency lock-in - we'll be better positioned to migrate to or support other charting libraries in the future
3. Frees feature designers and developers from low-level implementation details, which increases feature velocity

Some initial potential abstraction needs we've identified:

1. Opinionated components and interfaces that implement/inject the OpenSearch Design System visualization guidelines (look and feel, interaction patterns)
2. Visualization theming interfaces (e.g. for consistent color assignment across multiple visualization types. See also #1165, #1241, #1466, #1775)
3. High-level React components for making common visualization creation easy, with minimal configuration
4. A single wrapper interface/integration point for annotating, extending, and otherwise modifying existing visualizations
5. Generic web components to empower server-side extensions
6. Guidance or contracts for plugins/features that use alternative rendering libraries
7. New visualization type plugin scaffolding/templates

Did we leave something out? Do you feel strongly about one of these areas or have solution ideas in these areas? Let us know in the comments and we'll update and expand the list.

Describe alternatives you've considered

Abstractions are not free; they carry a an additional cost to build, maintain, and document features that may already be available in the underlying charting library or rendering layer. But without an abstraction layer, all feature developers would need to learn how to effectively use the Vega-Lite grammar. They would also need to be familiar with the OpenSearch Design System, and be responsible for implementing it correctly and consistently.

Additional context

Much of the need for visualization abstractions came out of research into consolidating OpenSearch Dashboards visualization rendering to Vega-Lite.