superset icon indicating copy to clipboard operation
superset copied to clipboard

perf(sqllab): Rendering perf improvement using immutable state

Open justinpark opened this issue 2 years ago • 9 comments

SUMMARY

The main rendering in SqlLab is managed by the queryEditors state from redux(state machine). Each queryEditor state has been updated whenever an editor configuration has updated.

Since most components observes entire queryEditor state not specific attributes, any queryEditor updates trigger multiple repainting even if no visual changes impacted.

For example, when a user types any single character in the ace(sql) editor, it triggers the update for selectedText value in the active queryEditor state. This single change will trigger the following components repainted (though this change won't impact the left and bottom panels at all)

Screen_Shot_2022-07-26_at_3_10_16_PM

This unrelated repainting job causes the performance delay especially when table list is humongous.

Screen_Shot_2022-07-26_at_3_24_05_PM

The delay can be found in the following performance analysis due to the unnecessary repainting/redux post processes.

Screen_Shot_2022-07-25_at_4_08_35_PM

This commit fixes this performance issue by optimizing observers in each component to subscribe only related items. To minimize the regression issue from existing persistence state, this commit keeps the existing queryEditors state but make it immutable during the tab is active and only updates when the tab is switched. It introduces unsavedQueryEditor state which stores all these active changes.

With this update, the page rendering performance improved 3.5x faster(from 4.5s to 1.3s) and verified all unnecessary tasks cleaned.

Screen Shot 2022-07-25 at 4 18 45 PM

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before (4.5s):

https://user-images.githubusercontent.com/1392866/181118443-f8949bc1-1d07-4dcb-8a65-aec1ff024fee.mov

After (1.3s):

Screen_Shot_2022-07-25_at_4_19_14_PM

TESTING INSTRUCTIONS

  1. Go to SqlLab
  2. Choose a database and a schema that includes a large set of table list
  3. Open chrome console > Performance tab
  4. Select Caption settings on right menu
  5. Change CPU throttling to 6x slowdown
  6. type a string in the sql editor and then watch the refresh icon on the left panel (it will blink during repainting)

ADDITIONAL INFORMATION

  • [ ] Has associated issue:
  • [ ] Required feature flags:
  • [ ] Changes UI
  • [ ] Includes DB Migration (follow approval process in SIP-59)
    • [ ] Migration is atomic, supports rollback & is backwards-compatible
    • [ ] Confirm DB migration upgrade and downgrade tested
    • [ ] Runtime estimates and downtime expectations provided
  • [x] Introduces new feature or API
  • [x] Removes existing feature or API

justinpark avatar Jul 26 '22 22:07 justinpark

Codecov Report

Merging #20877 (dfb30fc) into master (8005b7f) will increase coverage by 0.07%. The diff coverage is 64.66%.

@@            Coverage Diff             @@
##           master   #20877      +/-   ##
==========================================
+ Coverage   66.27%   66.34%   +0.07%     
==========================================
  Files        1770     1772       +2     
  Lines       67543    67570      +27     
  Branches     7185     7187       +2     
==========================================
+ Hits        44764    44831      +67     
+ Misses      20940    20893      -47     
- Partials     1839     1846       +7     
Flag Coverage Δ
javascript 52.15% <64.66%> (+0.15%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset-frontend/src/SqlLab/App.jsx 0.00% <0.00%> (ø)
superset-frontend/src/SqlLab/types.ts 57.14% <ø> (ø)
...uperset-frontend/src/components/Dropdown/index.tsx 100.00% <ø> (ø)
...et-frontend/src/components/TableSelector/index.tsx 78.26% <ø> (+2.17%) :arrow_up:
superset-frontend/src/SqlLab/actions/sqlLab.js 61.37% <41.66%> (+1.26%) :arrow_up:
superset-frontend/src/SqlLab/reducers/sqlLab.js 34.85% <43.75%> (+1.52%) :arrow_up:
...frontend/src/SqlLab/components/SqlEditor/index.jsx 50.58% <47.05%> (-1.05%) :arrow_down:
...c/SqlLab/components/TemplateParamsEditor/index.tsx 82.35% <50.00%> (+7.35%) :arrow_up:
...et-frontend/src/SqlLab/reducers/getInitialState.js 46.66% <50.00%> (+0.23%) :arrow_up:
...d/src/SqlLab/components/SqlEditorLeftBar/index.tsx 50.00% <66.66%> (+0.79%) :arrow_up:
... and 17 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Jul 27 '22 00:07 codecov[bot]

@justinpark Ephemeral environment creation is currently limited to committers.

github-actions[bot] avatar Jul 27 '22 17:07 github-actions[bot]

/testenv up

ktmud avatar Jul 28 '22 00:07 ktmud

/testenv up

ktmud avatar Jul 28 '22 20:07 ktmud

@ktmud Ephemeral environment spinning up at http://34.220.154.64:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

github-actions[bot] avatar Jul 28 '22 20:07 github-actions[bot]

/testenv up

ktmud avatar Aug 02 '22 23:08 ktmud

@ktmud Ephemeral environment spinning up at http://34.217.34.22:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

github-actions[bot] avatar Aug 02 '22 23:08 github-actions[bot]

/testenv up

hughhhh avatar Aug 05 '22 17:08 hughhhh

@hughhhh Ephemeral environment spinning up at http://34.217.110.42:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

github-actions[bot] avatar Aug 05 '22 17:08 github-actions[bot]

@eschutho I updated the commit. Can you review the update? thanks!

justinpark avatar Aug 17 '22 23:08 justinpark

@eschutho @michael-s-molina Can we merge the PR?

EugeneTorap avatar Aug 23 '22 15:08 EugeneTorap

Ephemeral environment shutdown and build artifacts deleted.

github-actions[bot] avatar Aug 23 '22 15:08 github-actions[bot]