Implement as a library

[Cpasjuste]

Hi @juj,

I think it would be great to be able to build fbcp-ili9341 as a library. It would require a few changes thought, like splitting main loop in it's own function (something like FbcpInit, FbcpDeinit, FbcpProcessFrame). This would allow an easy integration directly into SDL (at least SDL2), preventing the use of hacks for vsync, polling etc (https://github.com/raspberrypi/userland/issues/440). I would then implement this in SDL2, something like:

FbcpInit();
...
eglSwapBuffers();
FbcpProcessFrame();
...
FbcpDeinit();

I will try to do it myself, but i guess it will require a lot more time than if you do it :)

Thanks, Cpasjuste.

[juj]

This is a great suggestion, something that I had mentioned in the TODOs in https://github.com/juj/fbcp-ili9341#i-want-to-contribute--future-work--todos . Unfortunately I can not at the moment take time to tackle this, so feel free to give it a go if you have some cycles and enough interest!

If EGL is present in the loop, i.e. with the proposed code:

eglSwapBuffers();
FbcpProcessFrame();

that does not avoid issue https://github.com/raspberrypi/userland/issues/440 , since EGL API lives around the GPU.

So something I would suggest is that for SDL applications that use software rendering, instead one would build a CPU-side "soft framebuffer" style of rendering backend to SDL, which would only host the current (CPU-side) framebuffer:

// eglSwapBuffers();
void *sdlFrameBuffer = SDL_GetFrameBuffer();
FbcpProcessFrame(sdlFrameBuffer, width*height*4);

What would be amazing with this kind of approach is that, especially if paired with Pi's GPIO-based input controller, it has the chance of reducing input-to-display latency to of game emulation to practically only few msecs ( https://www.reddit.com/r/retrogaming/comments/4yuftd/how_come_emulators_still_have_input_lag/ ), i.e. the time it takes to process a single game frame in the application. GPIO input lag is in nanoseconds, and pixels can be pushed to an SPI display instantaneously (since there is no vsync support), so e.g. with a 119Hz refreshing ILI9341 display, overall input-to-display latency would be < 1000/119 msecs - practically like original game consoles, although without vsync.

The idea of video https://www.youtube.com/watch?v=EOICdpjiqv8 was to investigate how good the latency would be with this kind of approach.

[Cpasjuste]

Hi @juj,

Yeah sorry, i'm a hobby dev (since a few years now..), so some stuff are still a little obscure to me :) But i think i do get it (and i could have before if my brain was working fine hehe). Calling eglSwapBuffers will request GPU to swap, but this is done async by the gpu, so we are in the same boat.

I will make changes to be able to use fbcp-ili9341 as a library, and i will start by writing a software renderer for SDL2, because it's easy to do so (i did that for the nintendo switch a few months/years ago, it's still fresh) and because i will focus on my FBA port (https://github.com/Cpasjuste/pemu/tree/master/pfba) which can already run with an SDL2 software renderer (minus shaders, but on a small 320*240 screen i guess it's not really needed). SDL2 software renderer is somehow slow, but if we don't do any scaling with SDL2, it will be as fast as possible.

This will take a few weeks as i have a lot of projects going on (and because i'm in vacation soon), but also because the device is in transit (freeplay cm3). I'll let you know how it's going, and probably ask for help/advice's as needed.

See you soon, Cpasjuste.

[Cpasjuste]

Ok, so, i did (quickly) implement fbcp as a library here: https://github.com/Cpasjuste/fbcp-ili9341/tree/library I do think that "fbcp" executable located here: https://github.com/Cpasjuste/fbcp-ili9341/blob/library/fbcp/fbcp.cpp will work as before if "FBCPInit" is called without arguments. Else, it will use provided framebuffer. I have a lot to say about this :)

• This is a quick, dirty and untested "work", and so, should not be merged.
• A lot of improvements are possible, like using arguments to FBCPInit to manage most options.
• The code was not tested yet, my (freeplay cm3) device just arrived in France, so i should get it soon and report :)
• @juj, your code is hard to read mate :) But you know that, and i usually work like you... my time is very limited either so no offence ;)
• I will update this post with other thoughts and idea, but right now i have to go!

[juj]

Sounds great - if you have access to a 120fps camera or similar, would be great to observe how input-to-display latency is affected. Likewise for CPU usage with top/htop. Such data would be useful for gauging impact of issue https://github.com/raspberrypi/userland/issues/440 .

[Cpasjuste]

Hi,

I did receive the (freeplay cm3) device a few days ago. Happy :)

So I was finally able to test my fbcp modifications and with a few fixes the main executable is working fine (except a crash on exit, will fix that asap). When using the (new) fbcp functions from my own sdl2 video driver, I do not get screen update. I think my sdl2 video code is correct, so I guess I missed something in fbcp main loop (fbcp_process()) for updating from my own framebuffer.

Else, even with the "standard" fbcp (with gpu etc), I did get pFBA to run very fine :) I don't know how the screen does without fbcp-ili*, but here its very fine, good job.

I'll take a closer look at the sdl2 video driver, probably tomorrow, I will keep you up to date here.

[zerog2k]

+1 for a merge-worthy way to get separate fb (want to have separate lcd status panel on front of rpi, which i can output to, separate from hdmi, which is driving TV with with a display manager on it.