tritonclient dependency change - cuda-python cause poetry installation failure in MacOS

[liusha-H]

trafficstars

Hi there

i got issue with poetry install mlserver, and it is due to a tritonclient 2.37+ now depends on cuda-python, it will block mlserver installation if the machine does not have NVIDIA GPU, wondering if it's possible to constrain the tritonclient version bit <2.37?

[adriangonz]

Hey @liusha-H,

Sure thing - sounds reasonable. Would you be willing to contribute a PR with the change?

[amirhessam88]

@adriangonz I am having the same issue; here is the tl;dr

I am running a ZenML pipeline with the following ZenML integration dependencies

mlserver = ">=0.5.3"
mlserver-mlflow = ">=0.5.3"

I get the following error running via python v3.9 on Mac M1 via poetry 👇

  • Installing cuda-python (12.2.0): Failed

  RuntimeError

  Unable to find installation candidates for cuda-python (12.2.0)

  at ~/.local/pipx/venvs/poetry/lib/python3.9/site-packages/poetry/installation/chooser.py:76 in choose_for
       72│ 
       73│             links.append(link)
       74│ 
       75│         if not links:
    →  76│             raise RuntimeError(f"Unable to find installation candidates for {package}")
       77│ 
       78│         # Get the best link
       79│         chosen = max(links, key=lambda link: self._sort_key(package, link))
       80│ 

Following @liusha-H 's suggestion fixed it 👇

mlserver = ">=0.5.3"
mlserver-mlflow = ">=0.5.3"
tritonclient = "^2.24,<2.27"

CC @RoyerRamirez

[privatedumbo]

@adriangonz , I can't help with this one. One doubt from my end. Why does MLServer need tritonclient in the first place?

[adriangonz]

Hey @privatedumbo, I'm no longer involved with this project - best would be to check with @ramonpzg on this one.

[ramonpzg]

Hi @privatedumbo -- Did you mean you can help with this one?

I believe the reason to have it there is improve communication with the triton server running alongside MLServer inside Seldon Core.

[privatedumbo]

@ramonpzg , I could, yes. For what I see, it's only used for batch-processing, so maybe it could be an extra dependency?