Bug: Houdini 20+ errors on launch due to `click` dependency

[BigRoy]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior:

Houdini 20+ on launch will show a bunch or errors on bookish which tries to run a flask server using click as one of its dependencies. However, it picks up the click dependency in OpenPype instead of Houdini's native click dependency.

See:

The newer version of click apparently exposes new attributes our older version does not have:

from click.core import ParameterSource

This is the failing line, see the log output below.

Houdini uses click version 8.1.6 OpenPype uses click version 7.1.2

We are likely on an older version because click 8.0+ dropped Py2 support.

Due to this error the Houdini native Help browser does not seem to work - other than that and the error it looks like Houdini does work fine, but of course there may be other obscure issues not found yet.

Expected Behavior:

The provided lib from OpenPype should not conflict with the one from Houdini so that Houdini can operate as normal.

We should either update our dependency to match (which is just us 'getting lucky'?) or avoid mangling dependencies if Houdini provides it?

Do we even need the click dependency from OpenPype/AYON in Houdini?

Version

3.17.6-nightly.3

What platform you are running OpenPype on?

Windows

Steps To Reproduce:

1. Configure Houdini 20.0.506
2. Launch Houdini from OpenPype
3. See Houdini console

Are there any labels you wish to add?

• [X] I have added the relevant labels to the bug report.

Relevant log output:

Installing OpenPype ...
Setting scene FPS to 25
Could not initialize the help server:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.9libs\houdinihelp\api.py", line 126, in initialize
    bookish_app = get_houdini_app(use_houdini_path=bool(hou))
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.9libs\houdinihelp\server.py", line 25, in get_houdini_app
    from bookish import flaskapp, flasksupport
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python39\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.9libs\bookish\flaskapp.py", line 37, in <module>
    import flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python39\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python39\lib\site-packages\flask\__init__.py", line 2, in <module>
    from .app import Flask as Flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python39\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python39\lib\site-packages\flask\app.py", line 34, in <module>
    from . import cli
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python39\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python39\lib\site-packages\flask\cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (C:\Program Files (x86)\OpenPype\3.17.4-colorbleed.1\dependencies\click\core.py)

Additional context:

No response

[cuID:OP-7416]

[BigRoy]

@iLLiCiTiT any ideas for best approach?

[iLLiCiTiT]

Oh crap, why it takes click from our dependencies?

[BigRoy]

Not sure where to look for that unfortunately. Haha.

[BigRoy]

Also, if we need click only in standalone/tray then I suppose we can also update it as well since that always run Py3? (Would still need to be removed from dependencies in hosts of course, but would allow updating it too)

[iLLiCiTiT]

Also, if we need click only in standalone/tray then I suppose we can also update it as well since that always run Py3? (Would still need to be removed from dependencies in hosts of course, but would allow updating it too)

It's actually needed all the time. Addons can add cli commands using click (it is required to import the addon). So it must be available in all DCCs.

[iLLiCiTiT]

Aaargh. I don't want to do wrapper around wrapper because a DCC is using different version 😢

[BigRoy]

It seem that we already have a special click for Py2 here: openpype\vendor\python\python_2\click So we should maybe just update the other and move it into the py3 folder?

Then Houdini might be able to function fine?

So the other click dependency should maybe be in py3 and remove it from AYON's startup global dependencies?

---

Not sure if this info is helpful:

[iLLiCiTiT]

So the other click dependency should maybe be in py3 and remove it from AYON's startup global dependencies?

I don't see a reason to move it to py3 dependencies? If we would update we don't need to move it to py3 dependencies.

We have older version to support Python 3.6 (resolve, fusion), so updating will break that, and I do expect we'll hit other issues in different DCCs after update.

[Minkiu]

Would pre-pending the Houdini libs path in sys.path ahead of the OP when in DCC work? So it picks that one before ours.

It's actually needed all the time. Addons can add cli commands using click (it is required to import the addon). So it must be available in all DCCs.

This is before launching the DCC, right? Or is to execute addons stuff within the DCC? Is there an example around?

[BigRoy]

Would pre-pending the Houdini libs path in sys.path ahead of the OP when in DCC work? So it picks that one before ours.

I suspect this would work yes - I've done something similar before in OpenRV at the time. A bit of a hack though, but should be doable. We'd need to make sure to reimport anything however that before the "switcheroo" was already imported because otherwise it'd still be picking the already imported module from the registered/imported modules.

This is before launching the DCC, right? Or is to execute addons stuff within the DCC? Is there an example around?

It's actually because the DCCs will also import the modules, and thus also import the part of the addon/module that defines the CLI (it basically imports that).

Anyway, that would still not fix the PIL issue itself

[iLLiCiTiT]

PIL should be also added to 3rd party dependencies (I thought it's already there).