floem icon indicating copy to clipboard operation
floem copied to clipboard

GPU texture override

Open jrmoulton opened this issue 1 year ago • 10 comments

Vello has support for overriding an image with a WGPU texture as a method of rendering a texture directly in scene.

I think we can do something simliar in vger.

To implement this I think we can just add a method to the Render trait called override_image that matches the vello::Renderer::overide_image and then leave it up to implementers of View to handle creating the image and calling the override.

@alexthegoodman

jrmoulton avatar Nov 13 '24 18:11 jrmoulton

@jrmoulton I think it'd be a great add. Although my wgpu integration was a bit more involved than rendering only a texture, I should say. I'm not sure how commonplace it is for people to want to integrate with the wgpu Surface, Frame, Resolve Frame, etc. but that's what my projects required.

alexthegoodman avatar Nov 13 '24 23:11 alexthegoodman

@jrmoulton I also had to re-pipe the GPU Resources through to pass into my app, so I had access to Device and Queue.

alexthegoodman avatar Nov 13 '24 23:11 alexthegoodman

+1 to this use case, I'm building an alternative to Substance Designer, and I'd like to be able to render textures to the screen directly from the GPU instead of wasting cycles pulling them back into the cpu

i'm using vello right now for that reason but i'd love to switch to something more high-level like floem

andi23rosca avatar Dec 07 '24 18:12 andi23rosca

@andi23rosca If you'd like to see how I achieved a full wgpu integration with Floem, you can take a look at my fork: https://github.com/alexthegoodman/common-floem It is not necessarily the cleanest, but it works great. You can find me integrating common-floem in my game engine here: https://github.com/alexthegoodman/midpoint-editor

Yes, for now it requires a fork and a fair bit of work, but I think the benefits outweigh that.

alexthegoodman avatar Dec 07 '24 18:12 alexthegoodman

@alexthegoodman Awesome! Forking floem and extra work is definitely easier than me reimplementing a view model in vello Will take a look, thanks 🙏🏻

andi23rosca avatar Dec 07 '24 20:12 andi23rosca

@alexthegoodman

if floem provided access to the device, the encoder, and an api for using a wgpu texture for a view, would that cover your use case?

wgpu Surface, Frame, Resolve Frame,

Why is it that you need direct access to the main window handle? (I've looked through your code but I don't totally understand it)

jrmoulton avatar Dec 08 '24 07:12 jrmoulton

@jrmoulton It would get us closer, I think. I use the Surface to load in my own shaders and build a separate Pipeline, and the Frames must be shared between the Floem pipeline and mine. (I also enabled multisampling by the way) If you check out these lines here: https://github.com/alexthegoodman/midpoint-editor/blob/master/src/main.rs#L984C9-L1004C93 You'll see how I am interacting with the window handle. I also set some handlers here: https://github.com/alexthegoodman/midpoint-editor/blob/master/src/main.rs#L1396-L1441 I might not need the full window handle anymore, but be aware that I am attaching an Any type to the window handle, so I can integrate any of my various projects: https://github.com/alexthegoodman/midpoint-editor/blob/master/src/main.rs#L55-L81 I think this is critical for sharing data around everywhere its needed.

alexthegoodman avatar Dec 08 '24 11:12 alexthegoodman

Any progress on this? I'd love to be able to render my textures straight to screen.

CoolSlimbo avatar Jun 17 '25 09:06 CoolSlimbo

There has been some progress. I've got a branch that puts the winit window, wgpu device, queue, and instance into the paint context. I need to do some work to clean it up. I'll try to get that pushed in the next few days

jrmoulton avatar Jun 17 '25 16:06 jrmoulton

Pr in #838

This still doesn't quite make it possible to render texture with vello because the vello specific api of texture overrides isn't exposed but this is a lot of the work. I'm not sure what the best way to expose the vello texture override is

jrmoulton avatar Jun 22 '25 05:06 jrmoulton