bevy icon indicating copy to clipboard operation
bevy copied to clipboard

Implement Auto Exposure plugin

Open Kurble opened this issue 1 year ago • 4 comments

Objective

  • Add auto exposure/eye adaptation to the bevy render pipeline.
  • Support features that users might expect from other engines:
    • Metering masks
    • Compensation curves
    • Smooth exposure transitions

This PR is based on an implementation I already built for a personal project before https://github.com/bevyengine/bevy/pull/8809 was submitted, so I wasn't able to adopt that PR in the proper way. I've still drawn inspiration from it, so @fintelia should be credited as well.

Solution

An auto exposure compute shader builds a 64 bin histogram of the scene's luminance, and then adjusts the exposure based on that histogram. Using a histogram allows the system to ignore outliers like shadows and specular highlights, and it allows to give more weight to certain areas based on a mask.


Changelog

  • Added: AutoExposure plugin that allows to adjust a camera's exposure based on it's scene's luminance.

Kurble avatar Mar 29 '24 23:03 Kurble

The generated examples/README.md is out of sync with the example metadata in Cargo.toml or the example readme template. Please run cargo run -p build-templated-pages -- update examples to update it, and commit the file change.

github-actions[bot] avatar Mar 29 '24 23:03 github-actions[bot]

This is really cool and you write great docs.

I had a look because I'm a bit interested in the histogram usage (#11468). Out of scope here but I'm hoping for a generalized histogram solution in the future for Bevy.

torsteingrindvik avatar Mar 30 '24 10:03 torsteingrindvik

Thanks for your feedback! I've made changes accordingly and fixed the CI.

A generalized histogram would be a bit more involved as the two compute shader passes rely on each other to work correctly, so there might be a slight performance hit if you want to separate them. I guess that's something to think about for a follow up PR?

Kurble avatar Mar 30 '24 11:03 Kurble

Thanks for your feedback! I've made changes accordingly and fixed the CI.

A generalized histogram would be a bit more involved as the two compute shader passes rely on each other to work correctly, so there might be a slight performance hit if you want to separate them. I guess that's something to think about for a follow up PR?

Agreed, future follow up for sure. At that point I'm hoping a visual histogram (like the graph here: https://github.com/bevyengine/bevy/pull/12561) would be relatively simple by just binding the already computed histogram and displaying it. But that's off topic here :slightly_smiling_face:

torsteingrindvik avatar Mar 30 '24 15:03 torsteingrindvik

Yep, none of my concerns are blocking here! I've committed the straightforward suggestions to reduce noise.

alice-i-cecile avatar May 02 '24 10:05 alice-i-cecile

Oh and @mweatherley, food for thought: This consumes the CubicGenerator from the math crate. First use of splines i've seen "in the wild", might be something to evaluate the curve trait PR on.

Yeah, it looks to me like this could ultimately just use any Curve<Vec2>, but I haven't looked extremely closely at it.

mweatherley avatar May 02 '24 11:05 mweatherley

I'll be addressing the doc things, I have some time for it. Latest merge with main unfortunately isn't working anymore so I'll be fixing that first! (Probably because of #13121 )

Kurble avatar May 02 '24 17:05 Kurble

@pcwalton you optimized out the exposure value that's in the ColorGradingGlobal in #13121, but the auto exposure shader is writing it's result to there. I looked for an alternative solution, but the auto exposure pass can't use the exposure value of theExtractedCamera, as that is used to render the scene before it's histogram can be constructed. So I just added an extra #else for when sectional color grading is not used.

Kurble avatar May 02 '24 18:05 Kurble

Very nice docs additions, thanks! Definitely wasn't necessary but it's great to leave the codebase just a bit nicer.

alice-i-cecile avatar May 02 '24 20:05 alice-i-cecile

Oh and @mweatherley, food for thought: This consumes the CubicGenerator from the math crate. First use of splines i've seen "in the wild", might be something to evaluate the curve trait PR on.

Yeah, it looks to me like this could ultimately just use any Curve<Vec2>, but I haven't looked extremely closely at it.

It could absolutely use a generic Curve<Vec2>, the only requirement is that the curve is monotonic on the x-axis. Could be a nice cleanup to use Curve::resample for the compensation curve texture once Curve has landed.

Kurble avatar May 03 '24 17:05 Kurble