Always use linear encoding in `Color`

[allenwp]

trafficstars

Describe the project you are working on

Godot engine

Describe the problem or limitation you are having in your project

Color may store linear values or nonlinear sRGB values. The problem is that it can be difficult to know if the data within Color is encoded as linear or nonlinear. Currently, I believe the only way to know this is from context, code comments, and function documentation, all of which may be unclear or missing.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Ensure data stored in Color is always linear encoded. This approach ensures that all math operations performed on color data will always be correct without needing to convert between encoding types.

Alternatively, this problem can be overcome by adding a new ColorLinear variant type and ensuring that data stored in Color is always nonlinear sRGB encoded.

Rationale

The primary purpose of nonlinear colour encoding is to reduce banding when limited memory is available for storing RGB values. This is relevant to 8-bit integer formats and not relevant to floating point storage formats, like what Color uses. Additionally, all math calculations on RGB color values should be performed in linear encoding to ensure correct results; the only time math calculations should be performed on nonlinear values is when memory, bandwidth, or performance limits data encoding to not use floating point formats.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Changes to Color

1. Replace all color constants with linear equivalents
2. Add automatic conversion from nonlinear sRGB (8-bit/16-bit integer codes) to linear (Color variant) and vice-versa for:
   • 8-bit properties
   • 8-bit constructors
   • hex function
   • hex64 function
   • html function
   • to_xxxx32 / to_xxxx64 functions
3. Remove linear_to_srgb and srgb_to_linear functions
4. Add new srgb_to_linear(r: float, g: float, b: float) function

Changes to editor

1. Update color picker to store values in nonlinear, but maintain user experience of picking colors in nonlinear sRGB
2. Update "RAW" to use linear RGB

Changes to scripting & existing projects

1. Oh boy, this is a big one. Probably needs to be a Godot 5 thing.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This is core.

Is there a reason why this should be core and not an add-on in the asset library?

This is core.

Feedback

Please provide notes on how you think this sort of change should be handled if it was to be implemented and I will update this proposal text. I'm sure there is a lot to cover with the implementation details in terms of gotchas and performance considerations...

[beicause]

Color may store linear values or nonlinear sRGB values. The problem is that it can be difficult to know if the data within Color is encoded as linear or nonlinear. Currently, I believe the only way to know this is from context, code comments, and function documentation, all of which may be unclear or missing.

In fact you can assume that all the Colors are in sRGB space at present. Except spatial shader and hdr 2d shader, there should be no place that uses linear space.

[fire]

I want this, but maybe the idea of treating Color as SrgbColor and inventing a new LinearColor might be compatibility feasible.

[allenwp]

Color may store linear values or nonlinear sRGB values. The problem is that it can be difficult to know if the data within Color is encoded as linear or nonlinear. Currently, I believe the only way to know this is from context, code comments, and function documentation, all of which may be unclear or missing.

In fact you can assume that all the Colors are in sRGB space at present. Except spatial shader and hdr 2d shader, there should be no place that uses linear space.

I think I understand where you're coming from. I made a comment on a docs PR about how documentation could help with this.

[allenwp]

I want this, but maybe the idea of treating Color as SrgbColor and inventing a new LinearColor might be compatibility feasible.

I think of adding a new Variant type to be a monumental effort, but it's probably a lot less work and risk overall than this proposal's approach. I might open a new proposal regarding adding ColorLinear once I've thought through what this could mean. In the end we might never do this, but I think there could be value in listing the places where a ColorLinear variant could be helpful so we can make an educated decision on whether it's worth all the effort and extra code/logic at some point in the future.

[fire]

I have a bad idea. We mark in colour an enumeration of colour space. like 4 floats to 5 floats which is a terrible idea.

[Mickeon]

It would be convenient if Variant had just enough memory space to store the color space, but it sounds like a really nasty solution.

Perhaps evaluation of a color space should be done by the GDScript compiler? Similar to how variables of enum types are elaborated. This also sounds quite taxing to implement, whatever form it would take, but it would alleviate the process quite nicely.

Either way I am not sure if this proposal is even a good idea. How does other software in general handle all of this?

[aaronfranke]

I have a better idea for how to solve this, but it requires adding a new feature to the GDScript compiler. Essentially it involves giving units to types, which would allow for a Color to be marked as linear or not, and a Vector3 to be marked as meters, or meters per second, or something else. Doing math would preserve units (m divided by s becomes m/s). Such units would only exist for compile-time checks, and would be optional. But suffice to say this idea would take a long time, for now a documentation improvement should suffice.

[allenwp]

How does other software in general handle all of this?

Typically a programming context is initialized with an encoding format. Or, if it is the global context, a single encoding format is used throughout the code as this proposal suggests. Before srgb_to_linear and such code, I suspect that Godot was like this with a single encoding format (nonlinear sRGB) throughout the code base.

The crux of the issue is that the scripting context of Godot is almost entirely nonlinear sRGB, but Color is also used with linear encoding through srgb_to_linear, so the simple rule of "Color is always nonlinear sRGB" is somewhat broken out of necessity to have linear encoded color values as well (for calculating luminance with get_luminance(), for example).

In terms of modern computer graphics that are not beholden to historical code, I believe floating point values hold linear values (for most uses including lighting and colorimetry calculations) or log encoded values (for color grading and perceptual calculations) Nonlinear sRGB has some similarities to log encodings in terms of its use, so if log encoding is not an option then nonlinear sRGB does have some value in certain operations.

[beicause]

How does other software in general handle all of this?

I made a quick search for Unreal. I found it has FColor which uses rgba8 for sRGB space, and FLinearColor which uses rgba32f for linear space . And there is a static lookup table for fast srgb8 to linear conversion. (linear to srgb8 is also a fast implemention)

Unity Color seems to same as godot Color, which is rgba32f, too, provides a linear propertyUnity Color. Unity also has a Color32 which uses rgba8.

(I have never used Unreal or Unity yet)

[allenwp]

Additionally, Unity has (or had?) a global project setting to switch between "gamma" and linear lighting calculations for 3D rendering. Unity's approach is complicated as it suffers from some of the same 2D/3D workflow challenges as Godot, combined with maintaining legacy workflows from ~2010. While it may be good to take inspiration from Unity because they have needed to tackle similar problems to Godot, the result is a system that is overcomplicated and difficult to maintain; it is equally important to learn from their mistakes.

[beicause]

I think it's not bad to add a 32-bit srgb Color32, and the current Color should only use linear encoding. Then we can replace the Color with Color32 as much as possible, which saves many memory. I believe there are many places that we don't need 4x32-bit Color.

[allenwp]

I think it's not bad to add a 32-bit srgb Color32, and the current Color should only use linear encoding. Then we can replace the Color with Color32 as much as possible, which saves many memory. I believe there are many places that we don't need 4x32-bit Color.

If this approach is used, I would favour terminology such as ColorNonlinear or ColorSRGB to hint that it exclusively holds nonlinear encoded values, while Color exclusively holds linear encoded values. Additionally, casting between the two would need automatic nonlinear/linear conversions to fully address the problem described in this original proposal.

I expect that I won't like any of the proposed solutions, including the one that I initially wrote, but I'm appreciating the thoughts on how this problem of ambiguity in Color encoding could be addressed.

[fire]

I like it. As far as I know variant’s colour is a float32 even in 64 bit builds.

~~To take advantage that Color32 (like near 0-1 and hdr plus) has much more bit precision than Color8 and use the linear encoding. So LinearColor with NonlinearColor being an alias of Color (current).~~

[fire]

Let me try this again.

Could be trivial to introduce a class alias for the 32 bit floating 0.0-1.0+ and change all of our codebase to use NonLinearColor or LinearColor. Color is required to be an alias of NonLinearColor. We can change our godot engine code internally to not use Color anymore.

[allenwp]

Besides nonlinear piecewise sRGB encoding, other working spaces could include log base 2, log base 2, scaled to middle grey of 0.18, normalized log2 and possibly some others. Again, these nonlinear encodings are good for color correction, grading, mapping to a LUT, etc. (desmos graph)

Nonlinear piecewise sRGB has additional benefits in 2D parts of the engine, partly for historical reasons, and partly in support of traditional 2D art workflows.

It might make sense for a game engine to dismiss the log encodings entirely in favour of piecewise sRGB. 😵‍💫

[fire]

// Base RGBA structure with 32-bit float components (0.0-1.0+ range)
typedef struct {
    float r;
    float g;
    float b;
    float a;
} NonLinearColor;

// Aliases for different color spaces/encodings
typedef NonLinearColor LinearColor;                     // Linear color space
typedef NonLinearColor LogBase2Color;                   // Log2-encoded color
typedef NonLinearColor LogBase2ScaledMiddleGreyColor;   // Log2 with mid-grey scaling
typedef NonLinearColor NormalizedLog2Color;             // Normalized log2 color
typedef NonLinearColor Color;                           // Primary alias (NonLinearColor)

We slowly change godot engine's code style from NonLinearColor (aliased as Color) to LinearColor.

[allenwp]

The problem is that it can be difficult to know if the data within Color is encoded as linear or nonlinear. Currently, I believe the only way to know this is from context, code comments, and function documentation, all of which may be unclear or missing.

A much simpler solution to this problem of ambiguity that can be applied immediately in Godot 4 is:

Make documentation explicit.

Currently, Color docs don't provide any explicit detail regarding encoding type. I'd like to hear others thoughts on my suggestion in this comment. Specifically, are there any user-facing scripting scenarios where Godot functions return linear encoded values without stating so in documentation?

[fire]

Can we do a search for color and colour in the documentation and submit prs? There's not that many ways to say rgbaf.

In my own personal opinion, the doc changes are straight-forward but other things.. we're probably need to drop one task's priority and I don't want to do that.

[allenwp]

Can we do a search for color and colour in the documentation and submit prs? There's not that many ways to say rgbaf.

Agreed, I'll start a PR separate from the one I linked sometime in the next week or so.