rendering to a virtual framebuffer

[lzunsec]

Hello, I'm a little lost on Wayland and this project. I see that it's possible to use winit, egl, etc. Is it possible to render to a virtual frame buffer? I'd like to put this thing inside a minimal VM with just networking access. I'd like to run a mimimum wayland application inside this VM using this project, and render it to a virtual framebuffer which would be sent over socket to host machine. This is just for learning purposes, of course, as this is a crazy idea.

[lzunsec]

I tried compiling this on an NVIDIA Jetson Nano and got

= note: /usr/bin/ld: cannot find -lxkbcommon /usr/bin/ld: cannot find -lxkbcommon /usr/bin/ld: cannot find -linput /usr/bin/ld: cannot find -lgbm

which I don't know how to install.

also I had to install libsystemd-dev libdbus-1-dev dbus-x11

Anyway, I wanted to run a minimal wayland server that renders to a framebuffer. No extra dependencies except for the rust ones.

[Drakulix]

Hey, sure thing you can render into a framebuffer.

The main question is, do you have any hardware-acceleration (that is OpenGL mostly) available inside your VM? If that is the case, you can do so with smithay very easily, you can instantiate a Gles2Renderer with the hardware available inside your VM, use that for rendering and read-out the pixels to copy them to your host system.

If you do not have any kind of hardware acceleration inside, you would need a software renderer to combine multiple wayland-surfaces of a single client into your framebuffer. Smithay does currently not provide one (as tracked by #330), but if you just want to display one application and no desktop, this might as easy as coping the buffer of the client-application into your framebuffer. So no real renderer is needed here, you would basically just do a memcpy.

which I don't know how to install.

This largely depends on the distribution you are running. But if you just want to do software rendering, you will probably not need all of those dependencies. Disable smithays default features and then slowly add, what you need.

[lzunsec]

It looks like the only example provided is of DRM. That is, it uses the first GPU available to render. I don't know to what it renders: a window, maybe? I also don't know where the socket is created so my wayland application can send its drawing commands to. Could you give me some context?

this might as easy as coping the buffer of the client-application into your framebuffer.

I think by this you meant completely ditching this repo and forcing my client to send its frame buffer to my frame buffer? Shouldn't I need a server to at least get the frames? I'm lost.

[Drakulix]

It looks like the only example provided is of DRM. That is, it uses the first GPU available to render. I don't know to what it renders: a window, maybe? I also don't know where the socket is created so my wayland application can send its drawing commands to. Could you give me some context?

That example code just covers the basic code necessary to initialize a backend, it does nothing on the wayland-server side. For a more complete example you need to take a look at anvil, which is a small compositor.

All the wayland-code to actually get the buffers would remain the same to what anvil currently does for your project, just the backend/rendering code would "compose" different client (so render them), but not display them using either the drm-backend or the winit-backend. Instead you would just copy your result (which is likely going to be a buffer of some sort) into your framebuffer.

So you would not ditch the whole repo, but most of its drawing code.

@vberger can likely give you some more insight on the wayland side of things.

[elinorbgr]

To add some more over @Drakulix's explanation, if you want to run a wayland client app you do need to implement the wayland protocol, which represents a significant amount of logic. Smithay provides helpers to handle a large part of it for you, but you'll still need to add some of your own logic to fit everything together, and notably to retrieve the contents of the buffers submitted by your clients and render them into your framebuffer.

The anvil directly of Smithay's tree provides an example compositor, and you may for example take inspiration from the winit backend, which is much simpler than the udev one and may be more similar to your case (with a single framebuffer), however if you only need to run a single app in your compositor, you can likely have a much simpler logic than that of anvil, as you don't really need a lot of window-management, so you can probably throw away most of the OutputMap and WindowMap structures for example.

Also, it looks like you are not very familiar with the general principles of wayland, so maybe you'd want to give a quick look to our (very WIP) smithay book? It only covers client-side basics at the moment, but that may give you a general idea of how a wayland client behaves (it is quite different from X11), which is quite helpful to have in mind when building a wayland compositor.