Distorted image in HTML5 client

[basilgello]

The 39c8838 + html5 master works fine but 31e258e produces distorted image on scroll either with xpra-org/xpra-html5@8cf87c2 or xpra-org/xpra-html5@ea8f02b

[basilgello]

Server log with HTML5 draw debugging enabled error.log

Encoding is Automatic with min-quality and min-speed set to 30.

[basilgello]

@totaam

[totaam]

This looks like a scaling issue. You have the decode worker disabled right? Can you try to re-enable it to see if that fixes the distortion?

[totaam]

My guess is that 31bef75d521a1247be151245947fa44dbb115f28 needs to be added to the non-decode worker code path. The longer term plan is to merge the two paths into a single support file so changes can be made in one place.

[totaam]

@basilgello does the commit above fix things? (for the non-decode worker code path)

If that's the case, then the remaining question is to figure out why the screen updates ended up using downscaling in the first place. Browsers are normally detected and tagged as such, and we should not use video mode unless it really looks like the same area is updating rapidly and constantly. I guess scrolling may be able to trigger that.

[basilgello]

@basilgello does the commit above fix things? (for the non-decode worker code path)

I tested it by copying the master tree into /usr/share/xpra/www and I dont see much difference :(

However, let me build two master trunks (xpra + html5) and test again.

[basilgello]

No, both trunks are the same as reported so it is not a fix

[basilgello]

I still dont understand why the master trunk of xpra still has this error:

2021-11-21 22:45:55,990 Error: failed to encode jpeg video frame using jpeg:
2021-11-21 22:45:55,990  YUV420P buffer is too small: 1120 bytes, 80x29=1160 bytes required
2021-11-21 22:45:55,991  source: ImageWrapper(YUV420P:(0, 0, 152, 29, 24):3_PLANES)

[basilgello]

And what's more interesting, breakages on scroll coincide with these,error messages in server.log.

[totaam]

@basilgello please clarify:

You have the decode worker disabled right? Can you try to re-enable it to see if that fixes the distortion?

---

I tested it by copying the master tree into /usr/share/xpra/www

That's the way I test too. Just make sure that you wipe this directory first because any existing pre-compressed files would take precedence.

I still dont understand why the master trunk of xpra still has this error:

"still" as in, you saw it before? I wasn't aware of this one, but anyway this is fixed: https://github.com/Xpra-org/xpra/commit/d75d53d9a0dc5a0f4484c9a9cdeac51a566bdd01

And what's more interesting, breakages on scroll coincide with these,error messages in server.log.

The failed to encode jpeg video frame? Then this should now be fixed.

[basilgello]

You have the decode worker disabled right?

Yes

Can you try to re-enable it to see if that fixes the distortion?

With xpra-org/xpra@d75d53d9a0dc5a0f4484c9a9cdeac51a566bdd01 and xpra-org/xpra-html5@master I could not reproduce screenshot number 1 either with DECODE_WORKER set to true or false.

What still remains is dim and blurry image that starts rendering on scroll event. In the logs, this corresponds to encode_yuv and goes away after encode_rgb

notice grey area while firefox headerbar is clear.

[basilgello]

"still" as in, you saw it before?

Yep, it is in the log in my 2nd comment in this issue:

Server log with HTML5 draw debugging enabled error.log

[totaam]

What still remains is dim and blurry image that starts rendering on scroll event. In the logs, this corresponds to encode_yuv and goes away after encode_rgb

That's exactly what subsampling to yuv does. The server uses subsampling because it detects a video region (the area that scrolls: fixed area with lots of updates) and so when it fails to encode it using scroll it will use a video encoder (or nowadays, a jpeg pseudo video encoder). Video mode uses csc subsampling and downscaling to save CPU, bandwidth and latency. When we get it wrong and the content isn't actually video, it is quite noticeable, especially on text which becomes blurry.

[basilgello]

Why video mode is triggered on if HTML5 client has "Video" advanced settong checkbox turned off?

[totaam]

Why video mode is triggered on if HTML5 client has "Video" advanced setting checkbox turned off?

Because those are only distantly related. The fact that this is not clear is good example of why this setting is not fit for purpose. Users need to be given clearer choices on the compromises they can make.

[basilgello]

That's exactly what subsampling to yuv does.

So it is intentional and there is no way to disable ot except ewverting to the working build? If you want I can send you the video showing the disastrous picture quality :(

[totaam]

Like I said before, I am working on this. Please be patient.

[basilgello]

Like I said before, I am working on this. Please be patient.

Not rushing you at all :) I am working on keyboard stuff too ;)

[basilgello]

Update on latest master trunk: I could not reproduce this issue running Firefox with LinkedIn, Youtube, etc a number of times. Encoding performance (3337) is still worse than xpra-org/xpra@39c8838 Should I package libspng for Debian and build Xpra using it?

[totaam]

I could not reproduce this issue running Firefox...

The fix is probably this one: https://github.com/Xpra-org/xpra/commit/35d030e2c7e05498a965aaf6d16d46e517aa423c#diff-ec21731672c0e9db9865626ceef607852d95efb34ed1bda017cd40b97fc6fe5c The correct fix was not to artificially increase min-quality but to figure out why the quality ended being too degraded. In this case, the jpeg quality values below ~20 were unwatchable (we now avoid using those - but only with jpeg), whereas for every other codec they are not.

Encoding performance (3337) is still worse

Please define what you're measuring and how. Having the -d compress log sample of the problematic screen updates would really help. What configuration and options? With or without the decode worker, other changes?

Should I package libspng for Debian and build Xpra using it?

At present this will make zero difference to the html5 client because only the decoder (https://github.com/Xpra-org/xpra/issues/3373) is actually used. I do intend to fix the encoder (https://github.com/Xpra-org/xpra/issues/3374), but how much of an improvement this is going to make over the plain libpng python-pillow encoder remains to be seen. I expect it to be faster just not so much faster that it will completely change the stakes.

Longer term though, it's probably a good idea as this will allow us to take advantage of https://github.com/Xpra-org/xpra/issues/3357 Talking about missing packages in Debian, the lack of libyuv is more of an immediate issue: the alternative ffmpeg's swscale module is good, but libyuv is faster.

[basilgello]

@totaam the latest master trunk of Xpra again introduced yuv subsampling on scroll making the stream quality non-acceptable. Reverting to 20211203 Xpra build restores the proper behavior (though still not as smooth as 2021110 but anyway)

[totaam]

Too late for 4.3 which was released today.

Please provide more details: reproducer, bisection or something i can use to narrow things down.

[basilgello]

Too late for 4.3 which was released today.

Next time please ping me on RC1 before the release :) Free QA wont hurt ;)

Please provide more details: reproducer, bisection or something i can use to narrow things down.

Let me capture the log first, then blame the commit :)

[mint-b]

Hi all, I am late to this comment but I just found it before I opened a ticket reporting the same issue. I have the latest beta build installed from 12/04/2022 xpra-html5-5.0-1.r1255.el8.noarch... and I get this issue with Firefox and some other applications.

Oddly enough this did not happen in the previous stable build. Only once I switched to beta so I could test the fix for issue #148 (which does work btw)

I am using CentOS Stream 8

Are there any logs or additional information that would be helpful to you? I am not sure what to provide at this point.

thanks! Best regards

[totaam]

Are there any logs or additional information that would be helpful to you? I am not sure what to provide at this point.

Steps to reproduce would be good. As per the comment above:

Having the -d compress log sample of the problematic screen updates would really help. What configuration and options?

Command lines, tweaks, etc.

[mint-b]

Hi,

To reproduce I: 1,) login to the HTML5 client through Google chrome 2,) On the top XPRA menu bar, "Start -> Internet -> Firefox" 3,) When Firefox opens, I just move my mouse around the Firefox window, or open the address bar by clicking on it and moving the mouse pointer around it.

The XPRA version that I am using is:

[root@centosstream8 xpra]# rpm -qa |grep xpra

python3-xpra-client-4.4-10.r31209.el8.x86_64
xorg-x11-drv-dummy-0.4.0-3.xpra1.el8.x86_64
python3-rencode-1.0.6-11.xpra1.el8.x86_64
xpra-common-4.4-10.r31209.el8.noarch
python3-xpra-4.4-10.r31209.el8.x86_64
python3-pyopengl-3.1.5-2xpra1.el8_3.x86_64
xpra-html5-5.0-1.r1256.el8.noarch
python3-xpra-server-4.4-10.r31209.el8.x86_64
x264-xpra-20211215-1.el8.x86_64
ffmpeg-xpra-5.0.1-1.el8.x86_64
xpra-common-server-4.4-10.r31209.el8.noarch
xpra-common-client-4.4-10.r31209.el8.noarch
xpra-4.4-10.r31209.el8.x86_64

[root@centosstream8 xpra]# uname -a
Linux centosstream8 4.18.0-373.el8.x86_64 #1 SMP Tue Mar 22 15:11:47 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

My XPRA config file is: 20220421_xpra_config_file.txt

Finally, as for the Logs, I am a bit new to this... I'm not entirely sure what you mean by... "Having the -d compress log sample of the problematic screen updates would really help." ... I reviewed the logs found in /var/log/xpra/proxy.log but not sure if it is what you are looking for...

20220421_XPRA_proxy_log.txt

...if not, how can I get the info that you need?

EDIT: I found that it seems like the problem originates in the encoding option section. When selecting Automatic or JPEG it blurs the windows, but selecting WebP or PNG resolves the issue but affects performance a bit

Thanks for your help. Best regards!

[totaam]

Superseded by #231