About an "InterpolatedMatrixTransform"

[KelSolaar]

Hi,

Starting an issue here so that we can discuss about an hypothetical future InterpolatedMatrixTransform. It also makes me think that there is maybe a case for introducing OEP(s): OpenColorIO Enhancement Proposals ala Python PEP, e.g. https://www.python.org/dev/peps/pep-0586/.

Anyway, the rational and context behind the InterpolatedMatrixTransform is as follows: We sometimes end up in situations where it would be convenient to be able to dynamically produce a new matrix as a function of a scalar parameter. For example, shameless plug and spoiler alert, we have shown at Siggraph 2021 that the camera specific calibration for a LED Wall is white balance dependent and a quick way to appreciate that is the following plot:

An IDT produced according to P-2013-001 is optimized for a given white-balancing colour
temperature. The resulting nx3 matrix defines the basis of an implicit RGB colourspace that
changes as a function of colour temperature. This can be observed in Figure 3.16, where the
implicit RGB colourspaces for Camera C are plotted for various colour temperatures.

One might understand how painful it is to handle all those transformations with OCIO and also that it is impossible to handle intermediate states.

With a reduced set of matrices, i.e. two, and a dynamic scalar parameter, it should be possible to express all those transforms with a single parameterized transform. @remia posited that it could be generalised and maybe have a way to blend any arbitrary transform. The scope for that might be bigger as we potentially need to render the two states and blend them together. With the suggested InterpolatedMatrixTransform, you really only need to generate a new matrix on the CPU thus it only requires rendering once.

Some other (related) use cases:

• Emulate what the DNG SDK does by interpolating for example D65 and Tungsten matrices as a function of white balance.
• Produce plausible albeit non-ideal intermediate IDTs.

Voila! Keen to hear people thoughts on this one.

Thomas

[doug-walker]

I think it's a good idea and would be useful. Although this technically could be added as a FixedFunctionTransform, it would require a lot of parameters (19) and it would be difficult to understand. So I agree this would be better as its own new Transform sub-class called InterpolatedMatrixTransform that takes a dynamic parameter for the blend.

Remi has an interesting idea, but I'm worried about inversion. If it's just a matrix, you blend the matrices first, then invert it, so it's simple. If it's two arbitrary transforms, you could not blend them into a single transform to then be inverted. So I don't think there is a closed-form solution.

[remia]

Yes the generalisation idea is probably not worth pursuing given the complexity and inversion issues you mention @doug-walker.

Another point I was wondering about is whether we could / should allow for more than 2 matrices in there, each associated with a scalar. That way you could get an actual range of value to support and allow for more complex use case like https://github.com/ampas/aces-dev/tree/dev/transforms/ctl/idt/vendorSupplied/arri/alexa/v3/EI800 or case where the LED Wall calibration would benefit from more than two matrices (although @KelSolaar plot seems to suggest 2 is fine if I understand correctly but that might be Wall / Camera dependent?).

[doug-walker]

@KelSolaar , I'd like to get more clarity on the scalar parameter for color temperature. In your diagram it seems to start out as values on the daylight locus and then switch to values on the black body locus. As you know, these locii are offset on a chromaticity diagram, so that seems like a discontinuous jump. Given that the daylight locus is only defined across a limited range of temperatures, perhaps it would be better to only use the black body (i.e., Planckian) locus? Would you like to make a more specific proposal regarding how the scalar parameter is defined? Thanks!

[KelSolaar]

Hi @doug-walker,

Sorry, this one slipped through the cracks! It is hard to give a good answer to this one as there is no real consensus on how this should be handled. It is typical I would say to use both locii and have a blend range, e.g. between 3500K / 4500K, where the illuminants are interpolated. It is maybe something @JGoldstone would be happy to contribute some wisdom. I know we talked about that a few times.

Now to go back to that particular transform, we should probably try to keep it as generic as possible. Having multiple matrices associated with their scalar weight as suggested by @remia should give enough flexibility to handle all the cases.

Cheers,

Thomas

[doug-walker]

Thank you Thomas, no worries!

Maybe a "temperature based" interface is not needed. It seems like the main requirement is an algorithm that takes chromaticity coordinates as input and produces the matrix interpolant value as output.

If anyone is able to refer us to any published work for blending between locii, that would be very helpful. Lacking that, It seems like it would be simplest to select a specific CCT algorithm and just use its locus rather than coming up with some non-standard blending between several locii.