citra icon indicating copy to clipboard operation
citra copied to clipboard

Frame presentation redesign

Open wwylele opened this issue 5 years ago • 4 comments

This is a mega issue about redesigning how game screens are rendered to the frontend. There are several issue around this and the solution to them are highly tangled together, as listed below:

  • We are using the deprecated Qt OpenGL widget for presenting right now;
  • Split window support needed. Read this as rendering to two screens at the same time = having multiple OpenGL contexts. This two problem leads to
  • Using new Qt OpenGL widget which supports multiple contexts, as attempted in #4737. However, it has a big problem, where it forces V-sync on on some platforms. This is related to
  • Decoupling guest V-sync and host V-sync is needed. The guest, 3DS, effectively has a constant V-sync at 60Hz, with respect to the emulated time, while the host can have different V-sync mode (unlimited, 60Hz, 144Hz, etc), all with respect to the wall clock. We currently relies on unlimited host V-sync (=no V-sync), which leads to tearing, and as said above, it is not guaranteed any more.
  • Split window also interacts with another two major thing. One is the screen layout configuration + side-by-side 3D. With two screens, there are more freedom and potential for custom layout. For example, one might want the upper screen window has side-by-side effect in two smaller windows to match the 3D monitor position while having an arbitrary size window. Our current window layout system might not be capable for this.
  • Split window also interacts with custom shaders (#4578). One might want to have different shader applied to different windows, or even to have the same screen duplicated with different shader effect applied.
  • The custom shader itself also has a problem right now, which is it needs custom vertex shader for some advanced filtering. We currently have a hard-coded vertex shader for presentation, and it has assumptions that make it inflexible.

Regarding these issues, I have some design idea:

The emulation side of the video core should renders to three intermediate textures (Left, right and bottom screens) at guest 60Hz. On the other side, we make a "post processing thread" which wakes up on host V-sync, render the three textures to arbitrary number of framebuffers/windows with configurable vertex/geometry/fragment shaders. The vertex/geometry shader handles custom layout, side-by-side 3D and the vertex part of custom shaders, and the fragment shader handles the fragment part of custom shaders.

wwylele avatar May 22 '19 22:05 wwylele