OCIO lut search path using environment variable

[dovanbel]

Hi,

I'm using OIIO on windows, built with python bindings. (release-2.0.11)

In my python script I'm using oiio.ImageBufAlgo.colorconvert(buf, buf, inColorspace, outColorspace, colorconfig=ocioconfig)

In the ocio config file, there is a parameter search_path which allows ocio to locate the luts used in the config.

If this search_path is set to 'lut', then everything works correctly, ie the oiio.ImageBufAlgo.colorconvert works as intended. However, if the search_path is set using an env var (in my case : luts/$EVENT) then the oiio.ImageBufAlgo.colorconvert method doesn't find the necessary luts and the image buffer is returned with no modification.

I have checked that the env var EVENT is correctly set. I suppose this is related to OpenImageIO, because when I use the same ocio config in Nuke, it works correctly. The OCIO search path finds the EVENT var, and searchs for luts in the correct folder

Regards

[lgritz]

I'm not sure. I don't think it's anything I'm specifically doing on the OIIO side. I've asked some OpenColorIO folks to take a look at this issue, hopefully one of them will have the proper advice.

[meimchu]

If you type "oiiotool --help", it will tell you the file path to which your config.ocio it is using. Can you verify if that's the same config.ocio file path that your Nuke is using as well?

[dovanbel]

I don't think I can use the oiiotool tool to check that, I'm using this in a Python script.

But I'm sure oiio is using the correct ocio config (meaning the same as in Nuke)

If the search_path in the ocio config file is :

$EVENT/luts

Then in Nuke the correct lut is applied, but nothing happens in OIIO (no colorspace transformation occurs)

If I hardcode the path in the search_path to MyExamplePath/luts

Then it works in Nuke as well as with OIIO, the same identical colorspace transform is applied

[hodoulp]

Is there any error/warning message(s) on the console?

[lgritz]

Any update, @dovanbel?

[dovanbel]

@hodoulp no, I don't see any error or warning message in the console.

@lgritz Sorry no, the only workaround so far is to avoid env vars in the search path, which is problematic in my case. If you have suggestions for other tests that I could do, I'd be happy to explore more.

[lgritz]

No, I'm perplexed, even after discussing it with the OCIO developers. OIIO just calls OCIO::Config::CreateFromFile(), either passing the explicit config filename you supply optionally, or else supplying the contents of $OCIO env variable. Then it just asks for a ColorProcessor to convert between the named color spaces and runs it on the pixels. That's it. It doesn't do anything that ought to affect how the internals of the ocio config are interpreted. And I don't know why Nuke would be different, assuming it and oiio are in fact reading the same config file and you don't inadvertently have two different configs (or don't change the $EVENT variable?).

The only other tests I can think of are:

1. Like I said try "oiiotool --help" just to verify that OIIO is reading the same config file as Nuke.

2. In your python script that is calling IBA.colorconvert, maybe right before that call, print out the value of os.getenv("EVENT") just to be sure it's what you think it is?

3. Check error values from oiio?

   ok = oiio.ImageBufAlgo.colorconvert(buf, ...)
   if not ok : 
       print("colorconvert error: " + buf.geterror())
   

[hodoulp]

@lgritz I checked IBA_colorconvert_colorconfig() & ColorConfig::createColorProcessor() and the code seems fine i.e. the OCIO errors are trapped & preserved. But there is a fallback to hard-coded 'color transformations' if the OCIO is not present or failed to create a processor. The latter condition is puzzling me.

• What if the OCIO processor failed with color space names recognized by the hard-coded path?

• What if user calls IBA_colorconvert_colorconfig() with an OCIO config file, but OIIO did not compile with the OCIO support?

Note: investigations done using RB-2.0

[lgritz]

The intent is to ensure that several common-sense color conversions can be relied on even if there is no OCIO support, or if there is OCIO support but no config is found, or if the OCIO config simply doesn't contain the requested color spaces. All of these happen in real life and frustrate non-color-scientist users who just want to correctly linearize an sRGB JPEG, say.

"What if the OCIO processor failed with color space names recognized by the hard-coded path?" -- I'm not sure what you mean by that. What are the circumstances in which OCIO is present, config file is found, config contains the color spaces requested, but it still fails?

I think if a config file is passed but OIIO was built without OCIO support, it does't even look for the file, it just falls back on the built-in color spaces.

[hodoulp]

The intent is to ensure that several common-sense color conversions can be relied on even if there is no OCIO support, or if there is OCIO support but no config is found, [...] All of these happen in real life and frustrate non-color-scientist users who just want to correctly linearize an sRGB JPEG, say.

Agree.

[...] or if the OCIO config simply doesn't contain the requested color spaces. [...]

The user selects a config file, but takes wrong color space names. Should that not be a error?

What are the circumstances in which OCIO is present, config file is found, config contains the color spaces requested, but it still fails?

One case could be to not find one single LUT file (i.e. the LUT filename attached to a color space definition is wrong). That could also happen if the search_path is wrong then no LUT files could be found. Could that be the problem here?

[dovanbel]

Hi,

I tested using the oiiotool. It works as expected, i.e. in a windows cmd console, I first set the environment variable, then I convert an image with the oiiotool and it will find the lut in the correct directory. I will now do some more testing using this within a python script.

[lgritz]

Well that's interesting. Hmmm.

Did you ever try the other tests I suggested? Is it possible that your python script is somehow changing the environment variable? Or, do you get any error returned from the colorconvert call?

[meimchu]

Yes please print the EVENT environment variable in your python script prior to calling oiio.ImageBufAlgo.colorconvert(buf, buf, inColorspace, outColorspace, colorconfig=ocioconfig) and see what that returns.

[dovanbel]

Hi,

Sorry for the delay : Test 1 : In my ocio config file I have: search_path: luts/$EVENT:luts

I inserted prints in my code just before the IBA.colorconvert command like this:

print ("EVENT env var is : ", os.getenv('EVENT'))
print ("CAMERA env var is : ", os.getenv('CAMERA'))
print ("ocio.inColorspace is : ", ocio.inColorspace)
print ("ocio.outColorspace is : ", ocio.outColorspace)
print ("ocioconfig is : ", ocioconfig)
oiio.ImageBufAlgo.colorconvert(buf, buf, ocio.inColorspace, ocio.outColorspace, colorconfig=ocioconfig)

The result is:

STDOUT: EVENT env var is :  EV103
STDOUT: CAMERA env var is :  AlexaV3LogC
STDOUT: ocio.inColorspace is :  AlexaV3LogC
STDOUT: ocio.outColorspace is :  ShotGrade
STDOUT: ocioconfig is :  \\sledge\vol1\Projects\Donat_test123_P42918\Compositing\Setups\OCIO\config.ocio

So it looks fine to me, ie it should work the EVENT env var is 'EV103', but unfortunately the conversion for AlexaV3LogC to Shotgrade is in fact not applied at all.

Test 2 I change the search path in the ocio config file to:

search_path: luts/EV103:luts

Then the resulting output is correct (correclty applied the lut) and the output of the prints is exactly the same.

I suppose I should do a self contained simple script + ocio config file so that someone else could also test it. Right now this is part of a post process script on a render farm.

[lgritz]

No idea how that would go wrong.

What happens with the failed colorconvert call? Does it change the pixels at all? Does it print an error? Does it return true or false? Does buf.geterror() contain anything afterwards?

[dovanbel]

Hi,

It does not change the pixels at all. The colorconvert call returns False. the buf.gerror() returns: The specified file reference 'Grade.cube' could not be located. The following attempts were made: \sledge\vol1\Projects\Donat_test123_P42918\Compositing\Setups\OCIO\luts\EVENT\Grade.cube : \sledge\vol1\Projects\Donat_test123_P42918\Compositing\Setups\OCIO\luts\Grade.cube

That's interesting. In the ocio config file the source path as I mentioned before is luts/$EVENT, but its looking into luts/EVENT...

[lgritz]

Well that is suspicious, isn't it?

I wonder if there is some code path internal to OCIO where for some reason it's botching the replacement of env variables in those strings? But for other cases it seems to work?

[lgritz]

Grasping at straws here, but what happens if you change luts/$EVENT to luts/${EVENT} ?

Also... I'm wondering if those windows paths with backslash separators are getting confused somehow. Are you mixing \ paths and / paths? (I don't know if that would even be a problem.)

[meimchu]

Are your two test results earlier from your own local computer or from the render farm computer logs? I am wondering whether there is a discrepancy between results from your local computer or render farm computers.

[dovanbel]

@meimchu I just tried this on my local machine, same error

[dovanbel]

@lgritz if I use ${EVENT} instead of $EVENT I get an error message too: error The specified file reference 'Grade.cube' could not be located. The following attempts were made: C:\Users\nozon\BTSync\OIIO_test\ocio\luts\{EVENT}\Grade.cube : C:\Users\nozon\BTSync\OIIO_test\ocio\luts\Grade.cube

[dovanbel]

Hi,

I have simplified my setup as much I could, and here it is :

NozCreatePreviewMovie_Debug.zip

In there you'll find: test_input.exr > an example test image ocio > folder containing the ocio config files and the luts. There are two config files: config_with_env_vars.ocio and config_without_env_vars.ocio. The two env vars are $EVENT and $CAMERA, so there's one config using the env vars, in the other I have replaced the env vars with actual correct values NozCreatePreviewMovie_Debug.py > the code using OIIO to read, convert and write the image. in this file, check lines 17 and 18. Here you can choose which ocio config file you want to use. On my system, if I use the config without env vars, I get the expected result (the output image is very orange). If I use the config with the env vars, the output image is the same color as the input image.

Could someone test this on it's system ? Thanks in advance

I didn't include my build of OIIO (which I built using vcpkg on the x64-windows platform with OCIO and Pybind).

[dovanbel]

Hi,

I got help from Simon Björk, which tested my script with two different builds of OIIO on windows

With his OIIO build with Python 2 bindings, he has the same problem as me. (ie the ocio config does not resolve the env vars) With his OIIO build with Python 3 bindings, there's no problem (the ocio config resolves the env vars like it supposed to, and then the luts are found)

Regards

[lgritz]

Well that is an interesting clue! But I still don't have any theories about how that could happen. OIIO isn't the one evaluating the line within the ocio config -- the read of the config and interpretation of the contents is strictly on the OCIO side, so I'm really scratching my head about how the OIIO python bindings could have any influence over that. Hmmm...

[lgritz]

The next thing I would try, if you are able to build OCIO from source, would be to find the place where it expands environment variables it finds in the config as it's reading, and print its values (and maybe the whole env table). I would be curious to understand if, by the time it gets into the OCIO code, it is somehow botching the parsing, the environment substitution, or if somehow the environment variables themselves have all been trashed before it even gets to that part of the code.

[lgritz]

No follow-up after the last comment, years ago. So closing.