web-stories-wp
web-stories-wp copied to clipboard
GoogleForCreators
Background Blur based on early trim PR
Summary
This version of Background Blur is based on the latest version of the new Trimming flow PR. The Media Pipe object is very difficult to cooperate and it often throws errors while switching between no effect and blur effect. In this PR I managed to make it work consistently. The recording part is not yet merged into this version (two media recorders one used to get the camera feed, the second for canvas recording when the blur effect is active).
Edges are now smooth so it looks a lot better than the first version:
Relevant Technical Choices
To-do
User-facing changes
Testing Instructions
- [ ] This is a non-user-facing change and requires no QA
This PR can be tested by following these steps:
Reviews
Does this PR have a security-related impact?
Does this PR change what data or activity we track or use?
Does this PR have a legal-related impact?
Checklist
- [ ] This PR addresses an existing issue and I have linked this PR to it in ZenHub
- [ ] I have tested this code to the best of my abilities
- [ ] I have verified accessibility to the best of my abilities (docs)
- [ ] I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
- [ ] This code is covered by automated tests (unit, integration, and/or e2e) to verify it works as intended (docs)
- [ ] I have added documentation where necessary
- [ ] I have added a matching
Type: XYZlabel to the PR
Fixes #11909
Plugin builds for 5878aaab2cd6cff0003382487bee37a4f26ac765 are ready :bellhop_bell:!
- Download development build
- Download production build
#12022 has been merged, so this can be updated to main to better reflect that actual changes.
Summary of the latest updates:
You will see two useMediaRecorder instances in the code: mediaRecorder and canvasRecorder.
When all effects are disabled (default state) we will do the recording in the old way (mediaRecorder will be the currentRecorder) - this is more performant way than always redrawing the camera stream onto the canvas and recording the canvas (with and without effects) - so it will be as snappy as possible on slower devices when you don't use any effects.
I also fixed https://github.com/GoogleForCreators/web-stories-wp/issues/12033
Size Change: +15.4 kB (+1%)
Total Size: 2.7 MB
| Filename | Size | Change | |
|---|---|---|---|
assets/js/6660.js |
0 B | -101 kB (removed) | đ |
assets/js/wp-story-editor.js |
1.36 MB | +3.15 kB (0%) | |
assets/js/6753.js |
113 kB | +113 kB (new file) | đ |
âšī¸ View Unchanged
| Filename | Size |
|---|---|
assets/css/carousel-view-rtl.css |
702 B |
assets/css/carousel-view.css |
701 B |
assets/css/web-stories-block-rtl.css |
4.52 kB |
assets/css/web-stories-block.css |
4.56 kB |
assets/css/web-stories-embed-rtl.css |
318 B |
assets/css/web-stories-embed.css |
317 B |
assets/css/web-stories-list-styles-rtl.css |
2.36 kB |
assets/css/web-stories-list-styles.css |
2.39 kB |
assets/css/web-stories-theme-style-twentyeleven-rtl.css |
102 B |
assets/css/web-stories-theme-style-twentyeleven.css |
102 B |
assets/css/web-stories-theme-style-twentyfifteen-rtl.css |
251 B |
assets/css/web-stories-theme-style-twentyfifteen.css |
251 B |
assets/css/web-stories-theme-style-twentyfourteen-rtl.css |
287 B |
assets/css/web-stories-theme-style-twentyfourteen.css |
287 B |
assets/css/web-stories-theme-style-twentyseventeen-rtl.css |
288 B |
assets/css/web-stories-theme-style-twentyseventeen.css |
288 B |
assets/css/web-stories-theme-style-twentysixteen-rtl.css |
224 B |
assets/css/web-stories-theme-style-twentysixteen.css |
224 B |
assets/css/web-stories-theme-style-twentyten-rtl.css |
143 B |
assets/css/web-stories-theme-style-twentyten.css |
143 B |
assets/css/web-stories-theme-style-twentytwelve-rtl.css |
256 B |
assets/css/web-stories-theme-style-twentytwelve.css |
256 B |
assets/css/web-stories-theme-style-twentytwenty-rtl.css |
86 B |
assets/css/web-stories-theme-style-twentytwenty.css |
86 B |
assets/css/web-stories-theme-style-twentytwentyone-rtl.css |
326 B |
assets/css/web-stories-theme-style-twentytwentyone.css |
326 B |
assets/css/web-stories-widget-rtl.css |
482 B |
assets/css/web-stories-widget.css |
482 B |
assets/css/wp-dashboard-rtl.css |
657 B |
assets/css/wp-dashboard.css |
659 B |
assets/css/wp-story-editor-rtl.css |
737 B |
assets/css/wp-story-editor.css |
738 B |
assets/js/1512.js |
6.15 kB |
assets/js/1814.js |
7.45 kB |
assets/js/2090.js |
84.6 kB |
assets/js/4319.js |
34.9 kB |
assets/js/4422.js |
49.3 kB |
assets/js/5051.js |
92.1 kB |
assets/js/785.js |
68.1 kB |
assets/js/9796.js |
3.71 kB |
assets/js/carousel-view.js |
3.41 kB |
assets/js/chunk-colorthief.js |
2.64 kB |
assets/js/chunk-ffmpeg.js |
5.89 kB |
assets/js/chunk-focus-visible.js |
1.01 kB |
assets/js/chunk-getStoryMarkup.js |
12.3 kB |
assets/js/chunk-html-to-image.js |
4.5 kB |
assets/js/chunk-opentype.js |
96 B |
assets/js/chunk-react-calendar.js |
12.5 kB |
assets/js/chunk-react-color.js |
44.3 kB |
assets/js/chunk-web-animations-js.js |
14.6 kB |
assets/js/chunk-web-stories-template-0-metaData.js |
545 B |
assets/js/chunk-web-stories-template-0.js |
11.4 kB |
assets/js/chunk-web-stories-template-1-metaData.js |
539 B |
assets/js/chunk-web-stories-template-1.js |
9.61 kB |
assets/js/chunk-web-stories-template-10-metaData.js |
533 B |
assets/js/chunk-web-stories-template-10.js |
7.37 kB |
assets/js/chunk-web-stories-template-11-metaData.js |
539 B |
assets/js/chunk-web-stories-template-11.js |
9.09 kB |
assets/js/chunk-web-stories-template-12-metaData.js |
497 B |
assets/js/chunk-web-stories-template-12.js |
9.7 kB |
assets/js/chunk-web-stories-template-13-metaData.js |
525 B |
assets/js/chunk-web-stories-template-13.js |
7.4 kB |
assets/js/chunk-web-stories-template-14-metaData.js |
582 B |
assets/js/chunk-web-stories-template-14.js |
7.37 kB |
assets/js/chunk-web-stories-template-15-metaData.js |
544 B |
assets/js/chunk-web-stories-template-15.js |
9 kB |
assets/js/chunk-web-stories-template-16-metaData.js |
588 B |
assets/js/chunk-web-stories-template-16.js |
10.9 kB |
assets/js/chunk-web-stories-template-17-metaData.js |
539 B |
assets/js/chunk-web-stories-template-17.js |
9.2 kB |
assets/js/chunk-web-stories-template-18-metaData.js |
585 B |
assets/js/chunk-web-stories-template-18.js |
9.91 kB |
assets/js/chunk-web-stories-template-19-metaData.js |
501 B |
assets/js/chunk-web-stories-template-19.js |
10.8 kB |
assets/js/chunk-web-stories-template-2-metaData.js |
586 B |
assets/js/chunk-web-stories-template-2.js |
9.3 kB |
assets/js/chunk-web-stories-template-20-metaData.js |
548 B |
assets/js/chunk-web-stories-template-20.js |
9.01 kB |
assets/js/chunk-web-stories-template-21-metaData.js |
535 B |
assets/js/chunk-web-stories-template-21.js |
9.85 kB |
assets/js/chunk-web-stories-template-22-metaData.js |
525 B |
assets/js/chunk-web-stories-template-22.js |
7.83 kB |
assets/js/chunk-web-stories-template-23-metaData.js |
605 B |
assets/js/chunk-web-stories-template-23.js |
7.48 kB |
assets/js/chunk-web-stories-template-24-metaData.js |
516 B |
assets/js/chunk-web-stories-template-24.js |
11.7 kB |
assets/js/chunk-web-stories-template-25-metaData.js |
544 B |
assets/js/chunk-web-stories-template-25.js |
7.06 kB |
assets/js/chunk-web-stories-template-26-metaData.js |
601 B |
assets/js/chunk-web-stories-template-26.js |
7.27 kB |
assets/js/chunk-web-stories-template-27-metaData.js |
542 B |
assets/js/chunk-web-stories-template-27.js |
7.82 kB |
assets/js/chunk-web-stories-template-28-metaData.js |
532 B |
assets/js/chunk-web-stories-template-28.js |
9.07 kB |
assets/js/chunk-web-stories-template-29-metaData.js |
562 B |
assets/js/chunk-web-stories-template-29.js |
9.25 kB |
assets/js/chunk-web-stories-template-3-metaData.js |
539 B |
assets/js/chunk-web-stories-template-3.js |
8.42 kB |
assets/js/chunk-web-stories-template-30-metaData.js |
576 B |
assets/js/chunk-web-stories-template-30.js |
7.89 kB |
assets/js/chunk-web-stories-template-31-metaData.js |
502 B |
assets/js/chunk-web-stories-template-31.js |
10.3 kB |
assets/js/chunk-web-stories-template-32-metaData.js |
552 B |
assets/js/chunk-web-stories-template-32.js |
13.3 kB |
assets/js/chunk-web-stories-template-33-metaData.js |
492 B |
assets/js/chunk-web-stories-template-33.js |
9.07 kB |
assets/js/chunk-web-stories-template-34-metaData.js |
571 B |
assets/js/chunk-web-stories-template-34.js |
7.58 kB |
assets/js/chunk-web-stories-template-35-metaData.js |
566 B |
assets/js/chunk-web-stories-template-35.js |
8.91 kB |
assets/js/chunk-web-stories-template-36-metaData.js |
577 B |
assets/js/chunk-web-stories-template-36.js |
12.7 kB |
assets/js/chunk-web-stories-template-37-metaData.js |
528 B |
assets/js/chunk-web-stories-template-37.js |
6.71 kB |
assets/js/chunk-web-stories-template-38-metaData.js |
572 B |
assets/js/chunk-web-stories-template-38.js |
7.94 kB |
assets/js/chunk-web-stories-template-39-metaData.js |
588 B |
assets/js/chunk-web-stories-template-39.js |
8.08 kB |
assets/js/chunk-web-stories-template-4-metaData.js |
564 B |
assets/js/chunk-web-stories-template-4.js |
12.7 kB |
assets/js/chunk-web-stories-template-40-metaData.js |
557 B |
assets/js/chunk-web-stories-template-40.js |
10.2 kB |
assets/js/chunk-web-stories-template-41-metaData.js |
572 B |
assets/js/chunk-web-stories-template-41.js |
7.75 kB |
assets/js/chunk-web-stories-template-42-metaData.js |
521 B |
assets/js/chunk-web-stories-template-42.js |
7 kB |
assets/js/chunk-web-stories-template-43-metaData.js |
557 B |
assets/js/chunk-web-stories-template-43.js |
8.76 kB |
assets/js/chunk-web-stories-template-44-metaData.js |
584 B |
assets/js/chunk-web-stories-template-44.js |
11.1 kB |
assets/js/chunk-web-stories-template-45-metaData.js |
565 B |
assets/js/chunk-web-stories-template-45.js |
7.52 kB |
assets/js/chunk-web-stories-template-46-metaData.js |
531 B |
assets/js/chunk-web-stories-template-46.js |
5.22 kB |
assets/js/chunk-web-stories-template-47-metaData.js |
592 B |
assets/js/chunk-web-stories-template-47.js |
9.42 kB |
assets/js/chunk-web-stories-template-48-metaData.js |
555 B |
assets/js/chunk-web-stories-template-48.js |
9.09 kB |
assets/js/chunk-web-stories-template-49-metaData.js |
518 B |
assets/js/chunk-web-stories-template-49.js |
9.69 kB |
assets/js/chunk-web-stories-template-5-metaData.js |
556 B |
assets/js/chunk-web-stories-template-5.js |
9.94 kB |
assets/js/chunk-web-stories-template-50-metaData.js |
503 B |
assets/js/chunk-web-stories-template-50.js |
9.15 kB |
assets/js/chunk-web-stories-template-51-metaData.js |
526 B |
assets/js/chunk-web-stories-template-51.js |
10.4 kB |
assets/js/chunk-web-stories-template-52-metaData.js |
602 B |
assets/js/chunk-web-stories-template-52.js |
10.4 kB |
assets/js/chunk-web-stories-template-53-metaData.js |
553 B |
assets/js/chunk-web-stories-template-53.js |
5.78 kB |
assets/js/chunk-web-stories-template-54-metaData.js |
547 B |
assets/js/chunk-web-stories-template-54.js |
7.67 kB |
assets/js/chunk-web-stories-template-55-metaData.js |
573 B |
assets/js/chunk-web-stories-template-55.js |
7.13 kB |
assets/js/chunk-web-stories-template-56-metaData.js |
542 B |
assets/js/chunk-web-stories-template-56.js |
9.87 kB |
assets/js/chunk-web-stories-template-57-metaData.js |
527 B |
assets/js/chunk-web-stories-template-57.js |
14.9 kB |
assets/js/chunk-web-stories-template-58-metaData.js |
555 B |
assets/js/chunk-web-stories-template-58.js |
5.74 kB |
assets/js/chunk-web-stories-template-59-metaData.js |
590 B |
assets/js/chunk-web-stories-template-59.js |
8.96 kB |
assets/js/chunk-web-stories-template-6-metaData.js |
569 B |
assets/js/chunk-web-stories-template-6.js |
7.07 kB |
assets/js/chunk-web-stories-template-60-metaData.js |
510 B |
assets/js/chunk-web-stories-template-60.js |
9.51 kB |
assets/js/chunk-web-stories-template-7-metaData.js |
569 B |
assets/js/chunk-web-stories-template-7.js |
7.46 kB |
assets/js/chunk-web-stories-template-8-metaData.js |
569 B |
assets/js/chunk-web-stories-template-8.js |
8.93 kB |
assets/js/chunk-web-stories-template-9-metaData.js |
581 B |
assets/js/chunk-web-stories-template-9.js |
8.46 kB |
assets/js/chunk-web-stories-templates.js |
845 B |
assets/js/chunk-web-stories-textset-0.js |
5.06 kB |
assets/js/chunk-web-stories-textset-1.js |
6.65 kB |
assets/js/chunk-web-stories-textset-2.js |
7.65 kB |
assets/js/chunk-web-stories-textset-3.js |
15.1 kB |
assets/js/chunk-web-stories-textset-4.js |
4.15 kB |
assets/js/chunk-web-stories-textset-5.js |
5.47 kB |
assets/js/chunk-web-stories-textset-6.js |
5.28 kB |
assets/js/chunk-web-stories-textset-7.js |
10.2 kB |
assets/js/generateBlurhash.worker.worker.js |
1.1 kB |
assets/js/imgareaselect.js |
3.77 kB |
assets/js/lightbox.js |
550 B |
assets/js/tinymce-button.js |
2.85 kB |
assets/js/web-stories-activation-notice.js |
25.4 kB |
assets/js/web-stories-block.js |
18 kB |
assets/js/web-stories-embed.js |
20 B |
assets/js/web-stories-widget.js |
587 B |
assets/js/wp-dashboard.js |
64.1 kB |
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
When switching on/off the blur background on Firefox, the image/video shakes/blinks/presents a weird behavior. Video below:
https://images.zenhubusercontent.com/235435637/dfc6390c-3c79-4894-b438-3c89660bf4f4/2022_09_06_10_58_55.mp4
The dialog seems a little confusing. Why don't any of the dropdown have label / legend to explain what these are. Depending on the name of your webcam, it really might not be clear what is happening here.
Still not seeing this functionality working in safari. https://user-images.githubusercontent.com/237508/189669000-cdc04a62-a3f2-4c80-be5e-f0503eab0748.mov
Still seeing the permissions popup in firefox.
https://user-images.githubusercontent.com/237508/189669720-7634e4f5-1f6d-46ef-92ed-a7def99ab61e.mov
Still seeing the permissions popup in firefox.
What happens when you select the first camera again? What happens after you refresh the page and select the first/second camera?
That issue is P3 and won't be done now, but those answers will still be appreciated (here or in the corresponding ticket) since it's not happening for me on two machines ("remember permission" is not enabled).
The dialog seems a little confusing. Why don't any of the dropdown have label / legend to explain what these are. Depending on the name of your webcam, it really might not be clear what is happening here.
I guess there was no design/UX on this. I'm also not a fan of inputs without labels (or rather one-time labels, until you select something).
Still not seeing this functionality working in safari.
Yep, Safari is tricky (when fetching frames from the canvas), still testing.
Noticed another weird issues with the settings dialog. See video.
https://user-images.githubusercontent.com/237508/189875958-87f31562-1bb6-4e20-bd56-6c2afd96c742.mov
Noticed another weird issues with the settings dialog. See video.
Screen.Recording.2022-09-13.at.11.14.03.mov
What browser is this? any errors in the console?
Seeing slow down of image in safari. https://user-images.githubusercontent.com/237508/190190745-4cd586d1-1e16-4403-ad02-b3ae0e38bd96.mov
And other issues in firefox. https://user-images.githubusercontent.com/237508/190190940-63efdbe5-bb73-4582-9cd9-bd60a170b504.mov
Seeing slow down of image in safari.
That is expected. It will never match the native blur or get close to it, tho it looks better on my Macbook Air.
And other issues in firefox.
~That is only happening on 2+ try in the same session, right?~ Probably fixed? PTAL
Safari I am seeing this error when I try active blur background.
Yes, it's WASM and some of its memory issues. I added a try/catch there that's the best we can do.
Yes, it's WASM and some of its memory issues. I added a try/catch there that's the best we can do.
More testing, I am seeing the slow version in safari. Blur background works but it is very slow in safari.
https://user-images.githubusercontent.com/237508/190210854-10d83e0a-ca09-4916-bff4-c07e94a49a9f.mov