iced icon indicating copy to clipboard operation
iced copied to clipboard

Add linear gradient support to canvas widget

Open bungoboingo opened this issue 1 year ago • 3 comments

GRADIENTS

This PR adds support for linear gradients (radial & conical TBD) as a Fill style for canvas widgets.

Please note this does NOT add support for gradients for quads (yet), only 2d meshes on a canvas.

Big thanks to @tarkah for letting me yoink his original linear-gradient wgpu PR to get my feet wet in Iced. 🤗

Known issues:

~1) When running examples with both a solid & gradient shader with the glow backend, only one shader type will be drawn until the window is resized.~ ~2) Rainbow example is broken, need to rework it 😢~ <- this was mutually agreed upon to be removed ~3) Declaring a color stop offset out of order (e.g. "0.3", "0.6", "0.2", then "1.0") causes the gradient to straight up not work~

Usage

When creating a primitive you can now change the FillStyle to be either Solid or a Gradient. This changes the original field from color to style. E.g.:

Before:

Fill {
    color: Color::BLACK,
    .. Default::default()
}

After:

Fill {
    style: fill::Style::Solid(Color::BLACK),
    .. Default::default()
}

//or!

let gradient = Gradient::linear([Point::ORIGIN, Point::new(bounds.width, bounds.height)])
    .add_stop(0.0, Color::from_rgb(1.0, 0.0, 0.0)) //color at the beginning of the gradient (red)
    .add_stop(0.5, Color::from_rgb(0.0, 1.0, 0.0)) //color half-way through the gradient (green)
    .add_stop(0.1, Color::from_rgb(0.0, 0.0, 1.0)) //color at the end of the gradient (blue)
    .build()
    .unwrap(); //invalid color stop locations will panic

Fill {
    style: fill::Style::Gradient(&gradient),
    .. Default::default()
}

Any direction of linear gradient can be created, with start and end points. Color "stops" are added as a percentage along the total length of the gradient, akin to making a linear-gradient with CSS. So a color stop with an offset of "0.5" would mean the color is fully rendered at 50% of the total length of the gradient. A color stop with an offset of "0.0" would mean the color is fully rendered at 0% of the total length, etc.

This builder was created by @tarkah (I yoinked it from his PR) and not sure that git will attribute it correctly, so big thanks to him!!

Implementation Details (Feedback appreciated!)

I am still fairly new to graphics programming, WGPU in particular, so any feedback would be appreciated! 🤗

Iced currently has no support for multiple shader types. In this PR I've attempted to modularize adding new pipelines (wgpu)/programs(opengl) so we can add more shader types easier in the future.

My first change is to remove color data from the current Mesh2D attribute data, leaving only position data as a Vertex2D. For gradients this attribute is unneeded and was wasting quite a bit of bytes in the attribute buffers, so it has been purged. For solids as well this wasn't really needed as a fill generally does not switch colors per-vertex, but was probably more efficient than adding a uniform write potentially every draw. I've switched all color information to being uniform based. Performance implications of this TBD.

Other areas that were changed:

  • Modularized the WGPU/Glow rendering backends to be a little more scalable and tried to remove as much code duplication as possible (within reason!)
  • Added different abstractions for buffer types in WGPU backend (static & dynamic, more can be added as needed e.g. a static uniform buffer)
  • Added a new dependency to WGPU backend called encase (crates.io). This ensures compile-time safety for alignment & padding requirements for all uniforms in accordance with the WebGPU specification. This crate is recommended by the WGPU team and used in other large projects using WGPU for easier data management (Bevy being the biggest one). It introduces a derived ShaderType trait which generates metadata to perform checks on. It also comes with its own CPU buffer implementation which will pad data on write, making it very easy to avoid alignment issues.

Known areas to be improved

I will be doing some heavy profiling to see bottlenecks, but there are some obvious areas of improvement that I know will increase performance without doing so.

  1. Add implementation for OpenGL using VBOs for OpenGL 3.1+
  2. Add another implementation using SSBOs for OpenGL 4.3+ 😢
    • Due to these minimum version requirement for these features currently the gradient implementation on OpenGL has a hard-coded limit of 16 color stops. Any additional stops added after this limit will be ignored.
    • In contrast, the technical color stop limit on WGPU is 524,288 stops. 😿 anyways. But still good to do now if there is a uniform that passes the 256 byte threshold in the future.
  3. We do not need to be writing to the GPU buffer every draw call for WGPU & Glow backends both; the only issue right now is that for a Mesh we are storing attribute data in a buffer that can stay the same exact size but have its content change (e.g. in an example of a single rectangle with a fill that is update to be a new rectangle with a different fill; the attribute data would be the same size, but the uniform data must be changed). Thinking about best way to guard against this.
    • This would involve putting the burden of calculating a diff on the CPU vs just rewriting every draw() on the GPU. Performance could be better in either situation depending on hardware. For now we are just leaving it as writing to the GPU every draw. ~4) Uniforms can be rearranged in WGPU to pack them more tightly; working on this now in addition to the known issue(s) above.~
  4. Creating an UBER shader instead of having separate shaders might be a performance increase. Will have to see how much of a bottleneck swaping shader programs is during profiling.
  5. Remove branching from both fragment shaders for better performance since the else case will get executed every time on the GPU unless it performs some compilation-time unrolling, which, to my knowledge after some small amount of research, seems pretty hit or miss.

Areas to improve usability

  1. Right now it's slightly awkward to make a gradient for a rectangle and have to match the start/end points to the bounds to get it to fill the whole rectangle. I'd like to implement something like deg(0) that you can do with CSS gradients to just have a gradient fill the primitive at the right angle. This would be fairly trivial and just involve a wee bit of math. Keywords like left top etc would also be useful and make this process less awkward.

Edit: I have added a way to use relative positioning, but it still has the requirement of a bounds to know what to position relative to. A tad more cumbersome than the CSS implementation where you know the bounds of the object beforehand so don't need to specify it, but otherwise might be more comfortable to use in certain use-cases. For quad support I believe this bounds check can be builtin.

let gradient = Gradient::linear(
    Position::Relative {
        top_left: Point::ORIGIN,
        size: bounds.size(),
        start: Location::TopLeft,
        end: Location::BottomRight
    }
)
    .add_stop(0.0, Color::from_rgb(1.0, 0.0, 0.0)) //color at the beginning of the gradient (red)
    .add_stop(0.5, Color::from_rgb(0.0, 1.0, 0.0)) //color half-way through the gradient (green)
    .add_stop(0.1, Color::from_rgb(0.0, 0.0, 1.0)) //color at the end of the gradient (blue)
    .build()
    .unwrap(); //invalid color stop locations will panic
  1. ??? Seeking feedback! What do you think?

bungoboingo avatar Sep 29 '22 18:09 bungoboingo

So an example of using custom geometry to interpolate color data without a canvas a la examples/Geometry is currently impossible to do with only attribute data in this branch. If this is a use case we want to continue to support, I could rework Mesh2D to essentially have two variants: one which uses a shader and only has position data stored in its attributes, or one which does not use a shader (I mean both are using a shader in the end, but the built-in "Shader" type I mean for fills) and has both position and color data stored in its attributes. This would be a decent overhaul of what I've done but not too bad. Anyone have any thoughts?

bungoboingo avatar Sep 30 '22 15:09 bungoboingo

This is ready for another look; I've addressed all the feedback! 😸

The only remaining optimization that I have left to do is to rewrite the gradient fragment shader to not use branching for faster ALU calcs, which I am still trying to figure out how to do 😅.

bungoboingo avatar Oct 07 '22 02:10 bungoboingo

I tested this on all platforms I have except for an ARM Windows OS & Rasp Pi 3 (mine is dead apparently).

On Rasp Pi 2/4 performance is same as examples on master. I can detect no regressions.

If there are no other comments I think this is ready to merge!

bungoboingo avatar Oct 14 '22 02:10 bungoboingo

Tested examples & code looks good. Ready 2 merge I think! Great changes (and I will fmt & clean up commits on my next PR.. sorry!)

bungoboingo avatar Nov 03 '22 16:11 bungoboingo

No worries! Great work here :tada:

hecrj avatar Nov 03 '22 17:11 hecrj