[D&D] Drag and Drop meta issue

[ashwin-pc]

Parent issue to track the different task for the Drag and Drop experience.

• Feedback: #2280

Tasklist:

Framework

• Drag and drop entrypoint #959
• State management #947
• Visualization service #952
  • Contributions #1274
  • Remove contributions #1705
  • Expressions #1275
  • Agg service/viz class research #1699
• Save and embed
  • Save visualization #1620
  • Add to Dashboard via saved object #1703
  • Add to Dashboard directly from dashboard #960
  • Add to Dashboard directly from visualization #2534
• Localization #958
• Testing
  • Initial unit tests #1704
  • Integration and E2E tests #961
  • Increase test coverage #2380
• Drag and drop #956
• Initial visualizations #957
  • Metric #1614
  • Datatable #1615
  • Bar #1616
  • Line #1617
  • Area #1618
  • Pie #1619

UI

• Add Topnav #949
  • Persistance #1913
• Add Index pattern selector #948
• Add breadcrumbs #1621
• Empty workspace polish #1684
• Field summary view #950
• Field search #951
• Viz style #953
• Viz toolbar #955
• Viz type selector #954

Meta

• Documentation #962
• Replace "wizard" with real name #1706
• Plan to release as beta/preview feature #1708

---

• Original Design: #896
• Proposal: #883

[seanneumann]

For reference the RFC: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/896

[seanneumann]

I'd like to make sure we start talking about two requirements:

• Supporting PPL
• Supporting multiple data sources.

We don’t need to implement them in the initial release, but should include them in the design so they will be incrementally added.

[ashwin-pc]

I'm currently working on the design for #1275 which will cover these two topics. But just to highlight the general approach, we will be using the expressions language already built into OpenSearch Dashboards to create a pipeline that a visualization will use to render given the various user inputs. Current visualizations also use the same flow.

A sample pipeline:

 opensearchDashboards                                                   // Retrieves global information for the dashboards query and filters
 | opensearch_dashboards_context query='QUERY' filters='[]'             // Retrieves Visualization specific information such as datasource and filters
 | opensearchaggs index='INDEX_ID'  aggConfigs='AGGREGATION_CONFIG'     // Performs aggregation
 | opensearch_dashboards_pie visConfig='VISUALIZATION_CONFIG'           // Renders visualization

The implementation details for each function in the pipeline is separate from the function calling it. This way we can swap out any section of the pipeline with a different datasource and/or different query engine to render the same visualization as long as the inputs and outputs to the other functions in the chain remain unchanged.

[kavilla]

[Groom]: We should update this issue with our 2.2 planned deliverables.

@ahopp @opensearch-project/opensearch-ux the current 2.2 planned deliverables aren't feature complete and should be indicated as such. So that it should be configurable to toggle this on and off.

[seanneumann]

Hey team, I'd like to propose a new name for this functionality. While drag and drop is a part of the story, we should give it a name that is more encompassing. Here are some thoughts:

1. Vis Builder
2. Vis Wizard
3. Vis Designer
4. Vis Maker
5. Data Explorer

Let's discuss.

cc: @elfisher @ashwin-pc @ahopp

[kgcreative]

I'm partial to:

1. Vis builder
2. Data explorer

[xeniatup]

1. Viz builder - because it explains what it does really clearly
2. Data Wizard - hints on ease of use and emphasizes the underlying data

[btzeng]

Vis Builder

[abbyhu2000]

Vis builder or vis maker make the most sense for me:)

[seanneumann]

The official proposed name is "Visualization Builder"

@elfisher do you want to sign off on this?

[elfisher]

The official proposed name is "Visualization Builder"

@elfisher do you want to sign off on this?

Sounds good. I see there is a lot of momentum in 2.4 for this project. Are we overall on track to finish the tasks in this meta in 2.4?

[ashwin-pc]

Unfortunately no, we have focused our 2.4 efforts to harden the feature and fix bugs. I'm also hoping to have it enabled by default in 2.4.

Here is what we will actually be shipping for 2.4

• #2380
• #960
• ~~#2534~~
• ~~#1913~~
• #1706
• Open bugs

Edit:

1. #2380: All major functional tests complete (Unit tests will be covered in 2.5)
2. #2534: Has blockers that are called out in the issue
3. #1913: Might be pushed to 2.5. Ran into an issue that that will take a little more time to resolve
4. #1615: @ananzh has picked this up as a part of #1560
5. #950: Also for 2.4
6. #2672: Also for 2.4

cc: @seanneumann

[ashwin-pc]

Hello all, I wanted to provide an update on the project given the upcoming 2.7 release and the latest status.

Upcoming release (2.7)

In the next release, we have planned the following features and improvements:

Highlights

1. Data persistence: We will now be persisting the data across refreshes, switching chart types and even chart specific information like color.
2. 2 way communication: The visualization types can now interact with the applicaation to set properties such as timeranges and filters.
3. Bug fixes: There are a lot of minor bugs that individually dont break the expereince of the app, but collectively make it a lot easier to use the tool

The detailed list of changes are below:

• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3157
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2763
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2762
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3482
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3647
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3134
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3025
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3276
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2868
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2643
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3650
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3124
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1881
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1880
• https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3429

Latest status

To highlight the progess of the project as it stands now, here are some of them:

• Completed: 30/35 Originally identified subtasks
• Closed issues: 157
• Bugs: Only 2 non blocking bugs will remain in the 2.7 release.

Historical timeline

• Original release with Metric Visualization in 2.2. Brand new with the foundations setup.
• Released Bar, Line and Area charts using the existing charting libraries in 2.3
• Significantly improved test coverage, Added Table Vis and Field summary views in 2.4
• Fixed a lot of bugs, enabled the feature by default and renamed it to VisBuilder across the app in 2.4
• 2.5 &2.6 releases were focussed on incremental improvements to the user experience, focussed mainly around app persistence