AY-7315_Extract Review and OIIO transcode failing to transcode media blocking publishes

[BigRoy]

Is there an existing issue for this?

• [x] I have searched the existing issues

Current Behavior:

Over time we've had annoying reports where studios were unable to publish their renders because the publishing system (due to user configuration) tries to transcode e.g. a Z-depth AOV layer which may only have a named Z or Y channel and has no R, G, B or A channels and hence we don't really know what layer to process.

Instead of this we could also implement detecting more cases, so that if RGBA is not found in the file that maybe we allow to fall back to channels:

• XYZ
• Y
• Z (usually for z-depth?)

Here's metadata output for a file that fails without this PR:

<nchannels>1</nchannels>
<channelnames>
<channelname>Y</channelname>
</channelnames>
<alpha_channel>-1</alpha_channel>
<z_channel>-1</z_channel>
<deep>0</deep>

It's an Arnold Z-depth render AOV.

Error logs

2024-11-03 00:43:01:  0: STDOUT: ERROR:pyblish.plugin:Traceback (most recent call last):
2024-11-03 00:43:01:  0: STDOUT:   File "C:\Users\work\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\pyblish\plugin.py", line 528, in __explicit_process
2024-11-03 00:43:01:  0: STDOUT:     runner(*args)
2024-11-03 00:43:01:  0: STDOUT:   File "C:\Users\work\AppData\Local\Ynput\AYON\addons\core_1.0.3\ayon_core\plugins\publish\extract_color_transcode.py", line 163, in process
2024-11-03 00:43:01:  0: STDOUT:   File "C:\Users\work\AppData\Local\Ynput\AYON\addons\core_1.0.3\ayon_core\lib\transcoding.py", line 1141, in convert_colorspace
2024-11-03 00:43:01:  0: STDOUT:     input_arg, channels_arg = get_oiio_input_and_channel_args(input_info)
2024-11-03 00:43:01:  0: STDOUT:   File "C:\Users\work\AppData\Local\Ynput\AYON\addons\core_1.0.3\ayon_core\lib\transcoding.py", line 1430, in get_oiio_input_and_channel_args
2024-11-03 00:43:01:  0: STDOUT:     raise ValueError(
2024-11-03 00:43:01:  0: STDOUT: ValueError: Couldn't find channels that can be used for conversion.
2024-11-03 00:43:01:  0: STDOUT: Traceback (most recent call last):
2024-11-03 00:43:01:  0: STDOUT:   File "C:\Users\work\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\pyblish\plugin.py", line 528, in __explicit_process
2024-11-03 00:43:01:  0: STDOUT:     runner(*args)
2024-11-03 00:43:01:  0: STDOUT:   File "<string>", line 163, in process
2024-11-03 00:43:01:  0: STDOUT:   File "C:\Users\work\AppData\Local\Ynput\AYON\addons\core_1.0.3\ayon_core\lib\transcoding.py", line 1141, in convert_colorspace
2024-11-03 00:43:01:  0: STDOUT:     input_arg, channels_arg = get_oiio_input_and_channel_args(input_info)
2024-11-03 00:43:01:  0: STDOUT:   File "C:\Users\work\AppData\Local\Ynput\AYON\addons\core_1.0.3\ayon_core\lib\transcoding.py", line 1430, in get_oiio_input_and_channel_args
2024-11-03 00:43:01:  0: STDOUT:     raise ValueError(
2024-11-03 00:43:01:  0: STDOUT: ValueError: Couldn't find channels that can be used for conversion.
2024-11-03 00:43:01:  0: STDOUT: !!! ERR: 2024-11-03 00:43:01,066 >>> { CLI-publish }: [ Failed ExtractOIIOTranscode: Couldn't find channels that can be used for conversion. -- ('C:\\Users\\work\\AppData\\Local\\Ynput\\AYON\\addons\\core_1.0.3\\ayon_core\\plugins\\publish\\extract_color_transcode.py', 1430, 'get_oiio_input_and_channel_args', 'raise ValueError(') ] 

It's good to be aware that ExtractReview would also try to find a reviewable channel if it thinks it can't be processed by FFMPEG. Currently, if ExtractReview can't find a suitable RGBA channel then it'd just skip generating the reviewable for it.

However, ExtractOIIOTranscode runs before ExtractReview and would (or at least could?) usually perform the necessary conversion.

Expected Behavior:

Either:

• Allow the transcode to 'pass' through without an error but just a logged warning. OR
• Improve the channel detection to allow media to be much less unlikely to fail like this.

Version

1.0.0

What platform you are running on?

Windows

Steps To Reproduce:

1. Create a Maya Arnold render to EXR with separate AOV (not multilayer EXR) with a Z-depth AOV

Are there any labels you wish to add?

• [x] I have added the relevant labels to the bug report.

Relevant log output:

No response

Additional context:

We do have settings to "filter out" certain AOVs because we've had this issue before on Extract Review. The setting is at ayon+settings://deadline/publish/ProcessSubmittedJobOnFarm/aov_filter however it does not apply to ExtractOIIOTranscode because transcode may occur on something that isn't necessarily a reviewable (I suppose?)

[BigRoy]

Linking this prototype I once did to not raise an error but instead log a warning to at least allow continuing to publish: https://github.com/ynput/ayon-core/pull/988

[iLLiCiTiT]

I kinda do get grasp of the problematics, but I don't know full purpose of ExtractOIIOTranscode and if we need it for 3d renders. This https://github.com/ynput/ayon-core/pull/988 looks like it at least does not "crash", hard to tell if it is ok?

The proper solution would probably be, that maya would pass the information about output renders to publish job, which would require massive refactor of deadline addon and all host integration addons that are using deadline as instance and representation collection should be done in host integrations (because they know what the output will be) -> this would also require some changes in ayon-core. The refactor would also probably resolve issues connected to review AOV filtering. But that is epic.

[jakubjezek001]

I am also attaching here some info about SXR file format https://www.fxphd.com/fxblog/stereo-openexr-opensxr/. This is quite related to the post I was writing in discussion on our Discord thread.

I wouldn't be so sure about excluding non-RGB layers/files from OIIO processing. What if someone needs to convert EXR data passes into another format, or vice versa? I realize this is an unlikely situation, but we should remain open to possibilities and let the user decide how this could be used. Perhaps we should have some switch which would let user to decide on preset that it is only restricted to rgba or data.

[iLLiCiTiT]

I'm not sure user knows about it, he just sent renders to farm. Or I don't know if 3d artists do know what output it will create, I guess it is based on what renderer is used, and what is set in settings? In that case it is not that easy to resolve. But at the end, the plugin ExtractOIIOTranscode has 0% of information about it, we would still have to pass it to the publish job metadata file somehow, which we can't because we're guessing content of the files too. Maya addon has no way how to pass that information to publish job.

[philippe-ynput]

I have worked on that problem before. There are naming conventions originating from OpenEXR that can be used to support more than RGBA. https://openexr.com/en/latest/TechnicalIntroduction.html#channel-names

[BigRoy]

What if we add an additional filter to the transcode settings that is like a regex or list of channels to transcode if present - if no matching channels, leave the file as is?

[philippe-ynput]

Sure, but I think we should be able to handle single-channel files (A, Z or Y).

[BigRoy]

Yup - which would then be what we'd add to the 'defaults' in the profiles?

It does leave the question whether you really want to "transcode" a Z channel which is usually a data channel and not in an image rendered in a colorspace that needs colorspace conversions?

[philippe-ynput]

mmm... yes, if we want to keep things simple we should probably ignore data channels, like Z.

how would you present the new transcode settings ?

[['R', 'G', 'B', 'A'], ['r', 'g', 'b', 'a'], ['red', 'green', 'blue', 'alpha']]

[BigRoy]

Just wanted to flag this that came up on Discord here that mentioned using --if and --endif arguments to oiiotool to make it explicit 'optional' behavior directly on the CLI of OIIO.

When you have a lot of different inputs into oiio (because Substance Painter outputs can be very different from each other), you sometimes have to deal with special cases. For example, the webp format does not support 1-channel images. But png does. So you could use these oiio parameters : --if "{TOP.nchannels < 3}" --ch R,G,B --endif (I think it replicates the single input channel to 3 output channels)

https://openimageio.readthedocs.io/en/stable/oiiotool.html#cmdoption-if

A workaround would be to use negative regex rules in Extract OIIO Transcode-->Profiles-->Product names, to stop conversion for some kind of textures, but then you have to know the names of all textures kinds that are 1-channel, which is really cumbersome.

AND

And also related to this one (and its comment) : https://github.com/ynput/ayon-core/issues/1003

[philippe-ynput]

Good point @BigRoy. That would be even better. To be tested !

While I'm here, it looks like we could add deep image support quite easily:

File: client/ayon_core/lib/transcoding.py
730:         if input_info["attribs"].get("deep", False):
731:             # flatten deep images: they are not natively supported by ffmpeg.
732:             oiio_cmd.extend(["--flatten"])
733: 

[BigRoy]

Just adding @jakubjezek001's note that we'll also need to support stereo EXRs and may need to specify the "view" in the transcoding.

@philippe-ynput for now - should we just add the channels choose in settings? Shall we do a quick call to discuss to get this one rolling?

[BigRoy]

The approach now will be:

• Add channel mapping settings, so the settings would define which channels it would process. If no matching channels, the image is skipped instead of raising errors.
• We may need separate channel mappings for R, G, B and A so that e.g. an image with just Y could be mapped to render a greyscale/luminance image.
• We may want to use fnmatch instead of regex in settings here because it may be easier to use for artists/admins. Especially because channel names tend to also include dots.

The new defaults would then include a channel mapping that would match the current behavior so it's backwards compatible, except for it not raising an error if the image does not contain any matching channel. We'd then just log a warning most likely.

Aside of the linked OpenEXR page on channel names these examples came up during the discussion:

• Renderman beauty is Ci and a, see here

The default display channels are 'Ci' and 'a' (beauty and alpha)

• Arnold Z-depth AOV renders as only Z channel, no RGBA present.

---

I need to create a mapping in Settings where someone can state what channels in an image should map to which channel.

Like being able to say:

• If RGBA is present, then R=R,G=G,B=B,A=A
• If RGB is present, then R=R,G=G,B=B,A=1
• If Z is present, then R=Z,G=Z,B=Z
• If Y is present, then R=Y,G=Y,B=Y

And most likely even being able to support a regex (or fnmatch), like matching anything that endswith .R It'd basically replace this logic with an explicit mapping: https://github.com/ynput/ayon-core/blob/df95e252fa18b9e3d068357506c73023c6373a2d/client/ayon_core/lib/transcoding.py#L320-L403

The question then becomes:

What's a decent SettingsField design for this?