ERROR: Could not find a version that satisfies the requirement iree-compiler-snapshot

[sergiuiacob1]

What happened?

Until today, my requirements.txt file had no issues installing iree libraries: pip3 install -r requirements.txt

requirements.txt:

-f https://github.com/google/iree/releases/tag/snapshot-20210728.428
iree-compiler-snapshot
iree-runtime-snapshot
iree-tools-tf-snapshot
iree-tools-tflite-snapshot
iree-tools-xla-snapshot

As of today, the error I'm getting is:

sergiuiacob@Sergius-MacBook-Pro ~ % pip3 install -f https://github.com/google/iree/releases/tag/snapshot-20210728.428 iree-compiler-snapshot --no-cache-dir
Looking in links: https://github.com/google/iree/releases/tag/snapshot-20210728.428
ERROR: Could not find a version that satisfies the requirement iree-compiler-snapshot (from versions: none)
ERROR: No matching distribution found for iree-compiler-snapshot
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/Users/sergiuiacob/.pyenv/versions/3.7.13/bin/python3.7 -m pip install --upgrade pip' command.

Steps to reproduce your issue

pip3 install -f https://github.com/google/iree/releases/tag/snapshot-20210728.428 iree-compiler-snapshot --no-cache-dir

Version information

python 3.7.13 pip3 22.0.4

[sergiuiacob1]

My constraints are that I have to use these jaxlib and jax versions:

jaxlib==0.1.61
https://github.com/google/jax/archive/refs/tags/jax-v0.2.9.tar.gz

[sergiuiacob1]

I've also had issues with jaxlib before – versions just not being available all of a sudden anymore. What's a safe way to install these versions and just make sure CI/CD doesn't start failing all of a sudden?

[stellaraccident]

Not sure if related but suspicious timing: https://github.com/llvm/torch-mlir/issues/1374#issuecomment-1251692931

Basically, all of these are nightlies with no guarantees, although we do publish "stable" releases to pypi. I don't think we are extending any special care to jaxlib at the moment and would need to look into it. My best advice for now is that if you depend on pinned stable nightlies and minor versions over a long time (you are at a year+), your best bet is to mirror them somewhere if not getting them from pypi.

[sergiuiacob1]

Not sure if related but suspicious timing: llvm/torch-mlir#1374 (comment)

Basically, all of these are nightlies with no guarantees, although we do publish "stable" releases to pypi. I don't think we are extending any special care to jaxlib at the moment and would need to look into it. My best advice for now is that if you depend on pinned stable nightlies and minor versions over a long time (you are at a year+), your best bet is to mirror them somewhere if not getting them from pypi.

The problem with pypi is that there's a storage limitation and I've seen jaxlib / iree versions just disappearing overnight from the pypi repository. The reason I chose to install using github links if because I tought they have unlimited lifespan.

[silvasean]

This appears to be the same issue as https://github.com/llvm/torch-mlir/issues/1374 -- feels like GitHub is canarying the new releases page so it shows up irregularly across projects or even within the same project. We should be able to share a solution.

[stellaraccident]

I'm not sure yet this is the same issue. Is the issue that jaxlib won't install? The error you show signs to imply that, and it seems to indicate that they dropped support for python 3.7?

(We can pull a Jax dev in and ask but I'd like to know that is the problem -- I know what iree does for this stuff but not Jax)

[silvasean]

The ERROR: Could not find a version that satisfies the requirement iree-compiler-snapshot (from versions: none) is the telltale sign -- it parsed the HTML and didn't find any links ("none")

[sergiuiacob1]

I've fixed this by pointing pip to the assets' links directly

-f https://github.com/iree-org/iree/releases/expanded_assets/snapshot-20210728.428
--extra-index-url https://github.com/iree-org/iree/releases/expanded_assets/snapshot-20210728.428
iree-compiler-snapshot
iree-runtime-snapshot
iree-tools-tf-snapshot
iree-tools-tflite-snapshot
iree-tools-xla-snapshot

I think this can be considered a permanent fix, right?

[silvasean]

That hardcodes a specific version, so it is not a fully general fix.

[silvasean]

We should be able to do something like this: https://github.com/llvm/torch-mlir/pull/1389

[sergiuiacob1]

That hardcodes a specific version, so it is not a fully general fix.

That's true, but for my specific use-case it gets the job done.

[ScottTodd]

Something definitely changed on GitHub's side, creating our own pip-compatible releases page SGTM.

Our Colab notebooks are also broken right now because cells like

!python -m pip install iree-compiler iree-runtime iree-tools-tf -f https://github.com/iree-org/iree/releases

are fetching older releases (20220902.254, when 20220922.274 is available):

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Looking in links: https://github.com/iree-org/iree/releases
Collecting iree-compiler
  Downloading iree_compiler-20220902.254-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (48.1 MB)
     |████████████████████████████████| 48.1 MB 84 kB/s 
Collecting iree-runtime
  Downloading iree_runtime-20220902.254-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
     |████████████████████████████████| 2.3 MB 37.9 MB/s 
Collecting iree-tools-tf
  Downloading iree_tools_tf-20220902.254-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (59.0 MB)
     |████████████████████████████████| 59.0 MB 27 kB/s 
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from iree-compiler) (1.21.6)
Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from iree-compiler) (6.0)
Installing collected packages: iree-tools-tf, iree-runtime, iree-compiler
Successfully installed iree-compiler-20220902.254 iree-runtime-20220902.254 iree-tools-tf-20220902.254

[dellis23]

The fix for torch-mlir has been merged in this commit. This generates a package index at https://llvm.github.io/torch-mlir/package-index/. Feel free to grab inspiration from that commit if you'd like to do something similar in another repo.

It works via github actions. I ended up creating a separate branch solely for the github pages content, which that github action pushes to.

[aaron-schneider]

@GMNGeoffrey Can we remove the "awaiting-triage" label?

[GMNGeoffrey]

I think https://github.com/iree-org/iree/pull/10594 should fix this. I think at the point we're hosting our own thing, we should consider making it a PEP 503 compliant index. I'm going to move that discussion to https://github.com/iree-org/iree/issues/9532