superset
superset copied to clipboard
perf(sqllab): Rendering perf improvement using immutable state
SUMMARY
The main rendering in SqlLab is managed by the queryEditors
state from redux(state machine).
Each queryEditor state has been updated whenever an editor configuration has updated.
Since most components observes entire queryEditor state not specific attributes, any queryEditor updates trigger multiple repainting even if no visual changes impacted.
For example, when a user types any single character in the ace(sql) editor, it triggers the update for selectedText
value in the active queryEditor state. This single change will trigger the following components repainted (though this change won't impact the left and bottom panels at all)

This unrelated repainting job causes the performance delay especially when table list is humongous.

The delay can be found in the following performance analysis due to the unnecessary repainting/redux post processes.

This commit fixes this performance issue by optimizing observers in each component to subscribe only related items.
To minimize the regression issue from existing persistence state, this commit keeps the existing queryEditors
state but make it immutable during the tab is active and only updates when the tab is switched. It introduces unsavedQueryEditor
state which stores all these active changes.
With this update, the page rendering performance improved 3.5x faster(from 4.5s to 1.3s) and verified all unnecessary tasks cleaned.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before (4.5s):
https://user-images.githubusercontent.com/1392866/181118443-f8949bc1-1d07-4dcb-8a65-aec1ff024fee.mov
After (1.3s):

TESTING INSTRUCTIONS
- Go to SqlLab
- Choose a database and a schema that includes a large set of table list
- Open chrome console > Performance tab
- Select Caption settings on right menu
- Change CPU throttling to 6x slowdown
- type a string in the sql editor and then watch the refresh icon on the left panel (it will blink during repainting)
ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in SIP-59)
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [x] Introduces new feature or API
- [x] Removes existing feature or API
Codecov Report
Merging #20877 (dfb30fc) into master (8005b7f) will increase coverage by
0.07%
. The diff coverage is64.66%
.
@@ Coverage Diff @@
## master #20877 +/- ##
==========================================
+ Coverage 66.27% 66.34% +0.07%
==========================================
Files 1770 1772 +2
Lines 67543 67570 +27
Branches 7185 7187 +2
==========================================
+ Hits 44764 44831 +67
+ Misses 20940 20893 -47
- Partials 1839 1846 +7
Flag | Coverage Δ | |
---|---|---|
javascript | 52.15% <64.66%> (+0.15%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
Impacted Files | Coverage Δ | |
---|---|---|
superset-frontend/src/SqlLab/App.jsx | 0.00% <0.00%> (ø) |
|
superset-frontend/src/SqlLab/types.ts | 57.14% <ø> (ø) |
|
...uperset-frontend/src/components/Dropdown/index.tsx | 100.00% <ø> (ø) |
|
...et-frontend/src/components/TableSelector/index.tsx | 78.26% <ø> (+2.17%) |
:arrow_up: |
superset-frontend/src/SqlLab/actions/sqlLab.js | 61.37% <41.66%> (+1.26%) |
:arrow_up: |
superset-frontend/src/SqlLab/reducers/sqlLab.js | 34.85% <43.75%> (+1.52%) |
:arrow_up: |
...frontend/src/SqlLab/components/SqlEditor/index.jsx | 50.58% <47.05%> (-1.05%) |
:arrow_down: |
...c/SqlLab/components/TemplateParamsEditor/index.tsx | 82.35% <50.00%> (+7.35%) |
:arrow_up: |
...et-frontend/src/SqlLab/reducers/getInitialState.js | 46.66% <50.00%> (+0.23%) |
:arrow_up: |
...d/src/SqlLab/components/SqlEditorLeftBar/index.tsx | 50.00% <66.66%> (+0.79%) |
:arrow_up: |
... and 17 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
@justinpark Ephemeral environment creation is currently limited to committers.
/testenv up
/testenv up
@ktmud Ephemeral environment spinning up at http://34.220.154.64:8080. Credentials are admin
/admin
. Please allow several minutes for bootstrapping and startup.
/testenv up
@ktmud Ephemeral environment spinning up at http://34.217.34.22:8080. Credentials are admin
/admin
. Please allow several minutes for bootstrapping and startup.
/testenv up
@hughhhh Ephemeral environment spinning up at http://34.217.110.42:8080. Credentials are admin
/admin
. Please allow several minutes for bootstrapping and startup.
@eschutho I updated the commit. Can you review the update? thanks!
@eschutho @michael-s-molina Can we merge the PR?
Ephemeral environment shutdown and build artifacts deleted.