AcademySoftwareFoundation
OpenColorIO Properties
This is a generic version of an extension which has been used internally at Boris FX, implementing #102.
A set of five properties describe the OCIO setup of the host: kOfxImageClipPropOCIOConfig: the OCIO config. kOfxImageClipPropOCIOColourspace: the colourspace of the input images kOfxImageClipPropOCIODisplay: the display transform kOfxImageClipPropOCIODisplayView: the view kOfxImageClipPropOCIOLook: the look (optional)
A few questions
- this is all host-side. Is there any desire for a plugin to be able to request its inputs are in a specific colour space (or to negotiate a mutally-supported space with the host), or is this simply not possible due to the general nature of OCIO?
- what defines the colour space of the output image?
- what meaning if any do colour space properties have for matte inputs?
- what meaning if any do the display-related properties have when rendering out and there is therefore no viewer?
"clips which are color managed by OCIO" makes a requirement that the host uses OCIO. Hosts which have their own colour management may want to use this OFX mechanism without themselves using OCIO. Maybe rephrase these comments to make the requirement be that the image's colour space (and its subsequent treatment in any viewer) can be defined by OCIO?
Thanks for reviewing this Phil, my comments are below.
- this is all host-side. Is there any desire for a plugin to be able to request its inputs are in a specific colour space (or to negotiate a mutally-supported space with the host), or is this simply not possible due to the general nature of OCIO?
This is possible but we didn't do that in our internal version. If the host is using OCIO, it could construct a pipeline to convert the pixels into a different working colourspace from the same config as requested by the plug-in. However, if the plug-in is using OCIO itself, then this can be done on the plug-in side if needed, and the plug-in can choose whether to pay the price of the conversion or not.
- what defines the colour space of the output image?
It's assumed to be in the same colourspace as the input image, but that doesn't make sense for matte or data outputs such as flow vectors. A solution would be to have plug-ins set kOfxImageClipPropOCIOColourspace on output clips, where it could be set to "Raw", or an empty string could signify that it should not be colour managed.
- what meaning if any do colour space properties have for matte inputs?
The host could not set these properties, or they could choose an OCIO role with raw: yes which causes no colour management to happen.
- what meaning if any do the display-related properties have when rendering out and there is therefore no viewer?
None.
"clips which are color managed by OCIO" makes a requirement that the host uses OCIO. Hosts which have their own colour management may want to use this OFX mechanism without themselves using OCIO. Maybe rephrase these comments to make the requirement be that the image's colour space (and its subsequent treatment in any viewer) can be defined by OCIO?
I suppose that's possible, but the various strings are defined in a specific OCIO config rather than being global. So a host which did not use OCIO itself would need to maintain (or generate on the fly) an OCIO config that matched its own colour management, so that the plug-in could load it. Actually, this makes sense for something like ACES where there is a standard OCIO config available and a host may implement ACES directly. Would that make sense if you were going to use this extension in Baselight?
- what defines the colour space of the output image?
It's assumed to be in the same colourspace as the input image
No, that won't do. Which one if there's more than one input (or more gnarly, in a Transition context going between two images of different spaces)? What if we're in a Generator context (with no inputs)?
A solution would be to have plug-ins set kOfxImageClipPropOCIOColourspace on output clips, where it could be set to "Raw", or an empty string could signify that it should not be colour managed.
Either way it has to be documented. The host has to either tell the plugin what space it wants to receive, or be told by the plugin what space it's getting. Otherwise there's no point having any colour info anywhere.
- what meaning if any do colour space properties have for matte inputs?
The host could not set these properties, or they could choose an OCIO role with
raw: yeswhich causes no colour management to happen.
Either way it has to be documented. What does no-properties-set actually mean? Does it mean the host doesn't know?
- what meaning if any do the display-related properties have when rendering out and there is therefore no viewer?
None.
OK that needs to be documented too. And if the intention is that these viewer properties are only to be used if the plugin has its own UI and should/must not affect the rendered image, that needs to be documented too.
"clips which are color managed by OCIO" makes a requirement that the host uses OCIO. Hosts which have their own colour management may want to use this OFX mechanism without themselves using OCIO. Maybe rephrase these comments to make the requirement be that the image's colour space (and its subsequent treatment in any viewer) can be defined by OCIO?
I suppose that's possible, but the various strings are defined in a specific OCIO config rather than being global. So a host which did not use OCIO itself would need to maintain (or generate on the fly) an OCIO config that matched its own colour management, so that the plug-in could load it. Actually, this makes sense for something like ACES where there is a standard OCIO config available and a host may implement ACES directly. Would that make sense if you were going to use this extension in Baselight?
Yes that's precisely what I was thinking. If this becomes the way OFX does colour management then we'd have to make up a config file that contains sufficient information to describe the colour spaces that Baselight will be using around the plugin. In which case it will be necessary to document that a plugin should be careful not to assume that the OCIO config is (a) always the same file path or (b) unchanging and never needs to be re-read.
(NB to avoid unexpected behaviour, Baselight currently always passes linearised images to and from OFX, thanks to a discussion with Bruno years back, unless the plugin implements our private extension to negotiate a known colour space)
Actually should the viewer properties really be set per-clip (indeed in the current header, per-input-clip)? Surely any viewer is effectively global (and needs to be accessible by a Generator plugin as well).
I have revised the OCIO properties based on our previous discussions, so that they now address a wider range of use cases. Notably:
- kOfxImageEffectPropSupportsOCIO is provided on the host and plug-in descriptors to negotiate use of OCIO.
- The config, display, view and look properties have been moved to the effect instance, as these will not change per clip and are now usable in a generator context.
- kOfxImageClipPropOCIOColourspace is now read/write and plug-ins need to set it on their output clip.
- kOfxImageClipPropOCIOPreferredColourspace allows a host or plug-in to set the preferred colourspace for a given clip.
@barretpj and others please take a look and let me have your feedback.
Looks better now thanks.
I don't know OCIO but I gather all the colour space names are locally-defined in the config file so there's no globally-agreed name for any globally-defined spaces at all, right? So if a plugin wants to say "I want my inputs in a Rec.1886 tone curve with Rec.709 primaries" the plugin needs to find a space with a definition close to this in the config, and use its name?
This means plugins need to be able to access the config before naming any colour spaces on their clips.
However kOfxImageEffectPropOCIOConfig is set on the effect instance, but kOfxImageClipPropOCIOPreferredColourspace says it's set on a clip descriptor (i.e. at clipDefine time) not in a clip instance. Hopefully that's just a typing error on one side or the other?
However kOfxImageEffectPropOCIOConfig is set on the effect instance, but kOfxImageClipPropOCIOPreferredColourspace says it's set on a clip descriptor (i.e. at clipDefine time) not in a clip instance. Hopefully that's just a typing error on one side or the other?
Good point. I could rework kOfxImageClipPropOCIOPreferredColourspace so that it's instead handled during kOfxImageEffectActionGetClipPreferences, which should work for input clips, then move the property to the clip instance so the host can set it on an output clip. Does that make sense, or can you see a better way for the host to specify a preference?
Yes I think that makes sense. Can't make it a host property, because in a host that doesn't use OCIO but instead fabricates an OCIO config at runtime, the config could well not be global, it could be per-project for example. So it has to be per-instance.
We should add a list of predefined colorspace "roles" that match up with OCIO roles tha provide generic names such as "linear", "srgb", "data".
These are the typical roles on OCIO: https://opencolorio.readthedocs.io/en/latest/guides/authoring/overview.html#config-roles
We should probably set up another meeting to hash out the OCIO spaces/roles vs "generic" spaces/roles thing.
So my interpretation is that in OCIO a role is a reasonably well-defined (but not guaranteed?) name which OCIO resolves to a real locally-defined OCIO colour space. The actual colour space is not specified, it could (I imagine) be time-specific depending on the upstream effects in the timeline, project-specific, host-specific, or indeed user-preference-specific. The intention is that code can request "scene_linear" and get something that is likely to be some form of scene-linear space. It can then use OCIO to lookup the precise space definition if it wants/needs to - or it can simply believe that it's probably close enough to what that role is supposed to mean.
We should probably set up another meeting to hash out the OCIO spaces/roles vs "generic" spaces/roles thing.
Just some notes without discussing the current proposal per se:
Looking at the encoding list here:
https://opencolorio.readthedocs.io/en/latest/guides/authoring/colorspaces.html
scene-linear display-linear log sdr-video hdr-video data
Maybe that is fine for first handshake: check host in current context in clip preferences
Examples for a non OCIO user plug-in:
- if host is log, maybe would ask for scene-linear - if a special plug-in wants to know more use another layer to deal with that - as discussed by Phil, seems this is where a suite comes in?
- if host is sdr/hdr in some cases would keep material in that space - at base level, sRGB 8 bit, etc fits in that bucket, in many cases happy with just gamma approximation here. I note that a lot of cameras now encode "YUV" with range -0.5 to 1.5 in practice, -0.25 for iPhone .mov for example... The conversion into integer 16 bit is supposed to account for that using black and white point for internal remapping into intermediate internal buffers in float here.. In practice 16b float is usually good for that. There are a few hosts that are 16b only (might remember wrong but I think HS-Art Diamant, Hitfilm might be internally 16b float although promoted to 32b for effects?)...
- there is a tiny use case which is thumbnails generation etc - (that render path, and other variations of some display window calling a render). Color picking is another special topic.
- For specialized applications like color matching, is useful to be in as close as possible to what user sees for processing space.. example: doing color match from a DVD to a new film scan for rerelease in 4K, the two inputs are in different color spaces to start. Similarly our texture projection stuff, want the main input in a colorspace and the texture uv as data. And our denoiser has some max difference thing it tries to optimize for that work better in perceptual space... We sometimes internally convert to gamma 2.2 linear images to be closer, but
- In case where there are values like threshold, will likely want to get linear so changing working space does not change the threshold meaning internally. Thinking of hosts that globally change everything when you switch see in SDR versus HDR... a keyer is an example.
- If I ask for something different than current host state, would like to return e.g. linear and have the host do the conversion back
- Need to flag here e.g. "data" versus others to bypass post-transform by host Sometimes an option here display "data" versus display "color render"... In a 32b linear usually not an issue, host does not remap the color
So my interpretation is that in OCIO a role is a reasonably well-defined (but not guaranteed?) name which OCIO resolves to a real locally-defined OCIO colour space. The actual colour space is not specified, it could (I imagine) be time-specific depending on the upstream effects in the timeline, project-specific, host-specific, or indeed user-preference-specific.
The list of roles is recommended and typical, but not guaranteed. However, I think you'd typically find "scene-linear", for example. Roles are well-specified and fixed in an OCIO configuration. For example, in the OpenColorIO v2 config for ACES (config-aces-reference.ocio), these are the roles and what color space they refer to:
roles:
aces_interchange: ACES - ACES2065-1
cie_xyz_d65_interchange: CIE-XYZ-D65
color_timing: ACES - ACEScct
compositing_log: ACES - ACEScct
data: Utility - Raw
default: ACES - ACES2065-1
reference: ACES - ACES2065-1
rendering: ACES - ACEScg
scene_linear: ACES - ACEScg
Thankfully, OCIO 2.2 and ACES have already solved this problem for us. Two ACES configs are built in, and there is an API around built-in colourspaces which means that any software using OCIO 2.2 can rely on the names in the more basic of those configs, the ACES CG Config (not to be confused with the ACEScg colourspace).
OCIO 2.2 Release: Converting To or From a Known Color Space
Summary of the available colourspaces in the ACES configs
To support this I would alter kOfxImageEffectPropSupportsOCIO to be a string X 1 with the following options:
kOfxImageEffectOCIOACESCG
kOfxImageEffectOCIOACESStudio
kOfxImageEffectOCIOCustomConfig
This allows host and plug-in to negotiate what strings are allowed in the various colourspace properties. The rules would be:
- If neither host nor plug-in set
kOfxImageEffectPropSupportsOCIO, there would be no colour management. - If either host or plug-in sets
kOfxImageEffectOCIOACESCG, all strings must be from the configocio://cg-config-v1.0.0_aces-v1.3_ocio-v2.1. - If both host and plug-in set
kOfxImageEffectOCIOACESStudio, all strings must be from the configocio://studio-config-v1.0.0_aces-v1.3_ocio-v2.1. - If both host and plug-in set
kOfxImageEffectOCIOCustomConfig, strings may be anything from the chosen config. The host is free to pass in anocio://URI as the config to use one of the built-in configs.
Hosts and plug-ins that implement ACES using their own pipeline could now interoperate fully, by including a mapping from OCIO ACES strings to their own equivalent constants. Plug-ins that simply want to get a linear input can set kOfxImageEffectOCIOACESCG then confidently request scene_linear, and if they want to output motion vectors they can use the data role.
If people think this is a good idea, I will write it up.
JP, that seems quite reasonable. I'd add kOfxImageEffectNonOCIOColorManagement (or such) to indicate that the host (or plugin?) supports color space strings, but is not using OCIO (so there is no config which the plugin can read). Or maybe you're handling that in kOfxImageClipPropOCIOConfig by returning an empty string?
Right, I am redoing the strings to be less OCIO-centric and will update the PR.
Good idea, maybe put in comment what it maps to if you are using OCIO
On 2/8/2023 7:58 AM, John-Paul Smith wrote:
Right, I am redoing the strings to be less OCIO-centric and will update the PR.
— Reply to this email directly, view it on GitHub https://github.com/AcademySoftwareFoundation/openfx/pull/103#issuecomment-1422728573, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANCVYUBAXUMCA3SNHPSTHDWWOYCLANCNFSM6AAAAAASU3NS3Q. You are receiving this because you commented.Message ID: @.***>
The latest changes should be OK to review now. In short, I reworked the properties to provide two alternative colour management styles based on the built-in ACES configs from OCIO 2.2, which should be possible to implement without needing to actually use OCIO.
Same as ofxGpuRender, should this be a separate header file instead of in imageEffects...?
Same as ofxGpuRender, should this be a separate header file instead of in imageEffects...?
Seems reasonable as it's optional. I'll do that in the next iteration along with whatever other changes are needed.
I agree too that the new built-in ACES configs make it easier to standardize colorspace names. Just for the sake of reference, here's the ticket I opened a while back regarding the issue of locating "sRGB" in custom configs: https://github.com/AcademySoftwareFoundation/OpenColorIO/issues/1682
I have revised the spec to be more self-contained. The Native colour management style works with a set of colourspaces defined in ofxColourspaceList.h, which was generated by processing the OCIO ACES Studio config using scripts/genColour. Hosts and plug-ins could implement the Native style without using OCIO. Some basic attributes of the colourspaces are included - IsData, IsSceneLinear and Encoding. Implementations which simply want to group colourspaces into scene linear, log and video could use these attributes to do that.
All this data is simply preprocessor definitions done in a very simple way and needs supporting code to make it usable. I would encourage this to be contributed by someone who will actually use the native style - all our plug-in and host code uses OCIO.
The mechanism for specifying a preferred colourspace has been extended to allow the plug-in or host to specify an ordered set of preferred colourspaces. This is still a preference though, images could be provided in any colourspace and the receiver of an image must check the relevant property.
All this data is simply preprocessor definitions done in a very simple way and needs supporting code to make it usable. I would encourage this to be contributed by someone who will actually use the native style - all our plug-in and host code uses OCIO.
I'll be implementing it in a highly-colour-space-aware but non-OCIO host, though we don't use the host support library so I'm not sure whether contributing code based on that experience would be very useful?
Message ID: @.*** com>
Maybe nothing specific is needed, let's see if anything becomes obvious during your implementation.
Which of the many strings in ofxColourspaceList.h need to be used? For example the well-known ACEScct space would appear to be represented by kOfxColourspaceRoleColorTiming, kOfxColourspaceRoleCompositingLog, kOfxColourspaceRoleTexturePaint, kOfxColourspaceACEScct, kOfxColourspaceACEScctAliasACESACEScct and kOfxColourspaceACEScctAliasAcescctAp1. Does a host or plugin have to list all six of them?
the list is long and has a lot of redundancy (pretty complicated for a generic plugin) in most cases here (I might have a few outliers), would be happy in general case with just having a form of getGamma() - where 1.0 could assume is linear gamma without having to do any form of parsing of preference hints..
Would suggest to add a 4th type: kOfxImageEffectPropColourManagementBasic The minimal function is return gamma and if 1, I assume it's linear I could use other things but that is the minimal Point is I want (except perhaps for Log case) just get the buffer from host in state it's in right now and return it the same... which is a bit more than NONE.
I am not clear how the list of configs can sort of be sorted in buckets, and how to restore gamma 2.2 out of something else than the config label...
I somehow understand about primaries, white point, piecewise curves like sRGB, log space... except perhaps for color matching/correction related stuff here, I really don't care much about that aside gamma and perhaps range (e.g. if it's 0 to 1)... I hate API that concatenates in a single string things and end up with collapsing into things like linear srgb...
and what about color param (color picker...)? :)
Pierre
Did you really mean to close this PR?
Would suggest to add a 4th type: kOfxImageEffectPropColourManagementBasic The minimal function is return gamma and if 1, I assume it's linear
Most colour spaces don't fit a gamma curve, so for a host to work out what number to report here would be both hard and meaningless. Linear is the only special case, as we discussed before.