libva
libva copied to clipboard
intel
vaCopySurfaceGLX produces corrupt image
I have an app that uses vaCopySurfaceGLX. On some systems it works OK but others it produces a highly contoured magenta & yellow coloured image.
The problem is evident on an i7 8700 (Coffee Lake) system with integrated graphics running Ubuntu 20.04.1 with mesa 20.0.8. However an old i5 661 (Iron Lake) and the same Ubuntu 20.04.1 works fine. The problem persists when using libva built from git master. I've also tried building the latest intel-vaapi-driver but no change.
I believe there is a bug in the function create_tfp_surface() when it creates a framebuffer by calling glXChooseFBConfig. If I add 'GLX_ALPHA_SIZE, 8,' to the attribute list the problem is fixed. This should be included as the GLX_RENDER_TYPE is set to GLX_RGBA_BIT.
I have attached a patch that corrects the problem: 0001-GLX-Specify-an-alpha-size-when-requesting-a-GLX-fram.patch.txt
I can confirm the problem on Ubuntu 20.04.1 with i5 750 and Radeon RX550. The attached patch fixes the problem.
I'm happy to help out but have some reservations:
- Some GPUs are unaffacted. This indicates to me that the real fault lies outside of libva, possibly with glX (most likely) or in the intel-vaapi-driver or kernel as these have hardware dependent paths.
- The patch supplied is insufficient as-is and needs to include more attributes from the target texture. This works around the current undefined behavior in glX and/or driver.
- I don't have, nor do I want, a github account to create a working tree to pull from.
- I'm happy to provide a git compatible patch incorporating the changes discussed in (2). I have incorporated these changes into my own app which has tested OK over several platforms.
Since mesa 18.x or higher allow_rgb10_configs is set to true by default ! Can you check with setting it to false ? But still I think your patch is a good solution, or we could iterate over the fbconfig and choose the proper one
I have confirmed the problem on Ubuntu 20.04.2 with an i5-7Y57 using integrated Intel HD Graphics 615. The attached patch fixes the issue without causing and I would like to see it pulled into the mainline.
