is py-itree available for Windows?

[vmajor]

hiq-python depends on py-itree, but py-itree does not appear to have a windows compatible release.

[ernop]

I think the internal problem is that py-itree is not available by default on windows.

https://github.com/juncongmoo/itree

Haven't tried hard but this is where I ended up.

[vo2021]

I think you have to use Mac or Linux.

[juncongmoo]

Please use pip install py-itree==0.0.17 to install it for windows.

[bansan85]

I just tried:

  File "...\pyllama-main\inference.py", line 5, in <module>
    from llama import ModelArgs, Transformer, Tokenizer, LLaMA
  File "...\pyllama-main\llama\__init__.py", line 5, in <module>
    from .model_single import ModelArgs, Transformer
  File "...\pyllama-main\llama\model_single.py", line 8, in <module>
    import hiq
  File "...\Python\Python310\lib\site-packages\hiq\__init__.py", line 57, in <module>
    from .tree import (
  File "...\Python\Python310\lib\site-packages\hiq\tree.py", line 10, in <module>
    from hiq import tree_func
  File "...\Python\Python310\lib\site-packages\hiq\tree_func.py", line 19, in <module>
    from hiq.node_utils import _c, _d
  File "...\Python\Python310\lib\site-packages\hiq\node_utils.py", line 11, in <module>
    from hiq.node import Node
  File "...\Python\Python310\lib\site-packages\hiq\node.py", line 17, in <module>
    Node = itree.Node
AttributeError: module 'itree' has no attribute 'Node'```

[Eugene-Bond]

Traceback (most recent call last):
  File "/Users/eugene/Work/ai/pyllama/LLaMADemo/webapp_single.py", line 7, in <module>
    from llama import ModelArgs, Transformer, Tokenizer, LLaMA
  File "/Users/eugene/Work/ai/pyllama/LLaMADemo/llama/__init__.py", line 5, in <module>
    from .model_single import ModelArgs, Transformer
  File "/Users/eugene/Work/ai/pyllama/LLaMADemo/llama/model_single.py", line 8, in <module>
    import hiq
  File "/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/hiq/__init__.py", line 57, in <module>
    from .tree import (
  File "/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/hiq/tree.py", line 9, in <module>
    import itree
  File "/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/itree/__init__.py", line 7, in <module>
    import _itree
ImportError: dlopen(/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/_itree.cpython-39-darwin.so, 0x0002): tried: '/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/_itree.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/_itree.cpython-39-darwin.so' (no such file), '/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/_itree.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

Doesn't work with M1 Macs. Even though the package pretends to be built with arm64 architecture. Tried out with several versions of itree, and even pythons - the same problem :(

[Eugene-Bond]

Builds locally without any problems. TypeError: type torch.cuda.HalfTensor not available. Torch not compiled with CUDA enabled. is another problem and is not related to py-itree

[juncongmoo]

@Eugene-Bond @bansan85 @vmajor To install itree in windows, there is a workaround:

• install cmake
• run the following command to build it locally:

pip install https://github.com/juncongmoo/itree/archive/refs/tags/tag-bf9f3aada064acf3ce4db6fc58ed2e744caee0a3.tar.gz

I will release a better version of solution soon. Thanks.

[bansan85]

Thanks. Just tried. Install with success but I still can't run inference.py

...\pyllama-main>py -3.10 inference.py --ckpt_dir ../7B --tokenizer_path ../tokenizer.model
Traceback (most recent call last):
  File "...\Python\Python310\lib\site-packages\itree\__init__.py", line 5, in <module>
    from . import _itree
ImportError: cannot import name '_itree' from partially initialized module 'itree' (most likely due to a circular import) (...\Python\Python310\lib\site-packages\itree\__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "...\pyllama-main\inference.py", line 5, in <module>
    from llama import ModelArgs, Transformer, Tokenizer, LLaMA
  File "...\pyllama-main\llama\__init__.py", line 5, in <module>
    from .model_single import ModelArgs, Transformer
  File "...\pyllama-main\llama\model_single.py", line 8, in <module>
    import hiq
  File "...\Python\Python310\lib\site-packages\hiq\__init__.py", line 57, in <module>
    from .tree import (
  File "...\Python\Python310\lib\site-packages\hiq\tree.py", line 9, in <module>
    import itree
  File "...\Python\Python310\lib\site-packages\itree\__init__.py", line 7, in <module>
    import _itree
ImportError: DLL load failed while importing _itree: Le module spécifié est introuvable.

...\pyllama-main> py -3.10 -m pip freeze
cachetools==5.3.0
certifi==2022.12.7
charset-normalizer==3.1.0
fairscale==0.4.13
fire==0.5.0
hiq-python==1.1.8
idna==3.4
numpy==1.24.2
psutil==5.9.4
py-itree @ https://github.com/juncongmoo/itree/archive/refs/tags/tag-bf9f3aada064acf3ce4db6fc58ed2e744caee0a3.tar.gz
pyllama==0.0.2
PyYAML==6.0
requests==2.28.2
sentencepiece==0.1.97
six==1.16.0
termcolor==2.2.0
torch==1.13.1
typing_extensions==4.5.0
urllib3==1.26.14

[Eugene-Bond]

@juncongmoo thanks. as said above, building from source and installing from built wheel works. The main problem with M1 is that pyllama could run on CPU only. Even forcing torch.set_default_device("mps") doesn't help as complex types are not supported :-(

  File "/Users/eugene/Work/ai/pyllama/src/llama/model_single.py", line 42, in precompute_freqs_cis
    freqs_cis = torch.polar(torch.ones_like(freqs), freqs)  # complex64
  File "/Users/eugene/Work/ai/pyllama/.venv/lib/python3.9/site-packages/torch/utils/_device.py", line 63, in __torch_function__
    return func(*args, **kwargs)
TypeError: Complex types are unsupported on MPS

Anyway, thanks for an interesting product and responsiveness!

[levitation]

According to https://github.com/juncongmoo/itree/issues/18#issuecomment-1471425948 you can currently use Python 3.11 to use itree under Windows. In my Windows machine indeed, itree installs successfully under Python 3.11 and works ok.

BUT Python 3.11 is still no good solution either. pyllama also depends sentencepiece and that package does not install under Python 3.11 yet.

[thistleknot]

oh dear, so what is the solution to _itree on windows? I suppose build it on linux (which I have), but kind of unfortunate. The documentation highlighting this might be helpful (i.e. build instructions for windows/linux with python reqs and what works)?

[daniel-kukiela]

I am getting (Python 3.10 on Windows 10):

Traceback (most recent call last):
  File "C:\Users\Daniel\AppData\Local\Programs\Python\Python310\lib\site-packages\itree\__init__.py", line 5, in <module>
    from . import _itree
ImportError: DLL load failed while importing _itree: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\Temp\LLaMA_quantization\quant_infer.py", line 1, in <module>
    import hiq, time
  File "C:\Users\Daniel\AppData\Local\Programs\Python\Python310\lib\site-packages\hiq\__init__.py", line 57, in <module>
    from .tree import (
  File "C:\Users\Daniel\AppData\Local\Programs\Python\Python310\lib\site-packages\hiq\tree.py", line 9, in <module>
    import itree
  File "C:\Users\Daniel\AppData\Local\Programs\Python\Python310\lib\site-packages\itree\__init__.py", line 7, in <module>
    import _itree
ModuleNotFoundError: No module named '_itree'

https://github.com/juncongmoo/itree/issues/18 is closed even if (IMO) not fixed. I also tried installing from sources with CMake following this: https://github.com/juncongmoo/itree#build-from-source-with-cmake - still the same error.

[ashishpatel26]

ModuleNotFoundError: No module named '_itree'

I am facing the similar error

Traceback (most recent call last):
  File "[c:\Users\ibm26\.conda\envs\llms\lib\site-packages\itree\__init__.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/site-packages/itree/__init__.py)", line 5, in <module>
    from . import _itree
ImportError: DLL load failed while importing _itree: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "[c:\Users\ibm26\.conda\envs\llms\lib\runpy.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/runpy.py)", line 187, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "[c:\Users\ibm26\.conda\envs\llms\lib\runpy.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/runpy.py)", line 110, in _get_module_details
    __import__(pkg_name)
  File "[c:\Users\ibm26\.conda\envs\llms\lib\site-packages\llama\__init__.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/site-packages/llama/__init__.py)", line 17, in <module>
    from .model_single import ModelArgs, Transformer
  File "[c:\Users\ibm26\.conda\envs\llms\lib\site-packages\llama\model_single.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/site-packages/llama/model_single.py)", line 7, in <module>
    import hiq
  File "[c:\Users\ibm26\.conda\envs\llms\lib\site-packages\hiq\__init__.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/site-packages/hiq/__init__.py)", line 59, in <module>
    from .tree import get_duration_from_hiq_string, get_graph_from_string, Tree
  File "[c:\Users\ibm26\.conda\envs\llms\lib\site-packages\hiq\tree.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/site-packages/hiq/tree.py)", line 9, in <module>
    import itree
  File "[c:\Users\ibm26\.conda\envs\llms\lib\site-packages\itree\__init__.py](file:///C:/Users/ibm26/.conda/envs/llms/lib/site-packages/itree/__init__.py)", line 7, in <module>
    import _itree
ModuleNotFoundError: No module named '_itree'

[mypicture]

I faced the same error. Is there any solution?