setup_env.py error on MacBook M3 - FileNotFoundError: [Errno 2] No such file or directory: 'models/BitNet-b1.58-2B-4T/config.json'

[steve8210]

Following directions to setup on MacBook Pro M3

python3 setup_env.py -md models/BitNet-b1.58-2B-4T -q tl1 INFO:root:Compiling the code using CMake. INFO:root:Loading model from directory models/BitNet-b1.58-2B-4T. INFO:root:Converting HF model to GGUF format... ERROR:root:Error occurred while running command: Command '['<omitted>/BitNet/.venv/bin/python3', 'utils/convert-hf-to-gguf-bitnet.py', 'models/BitNet-b1.58-2B-4T', '--outtype', 'tl1', '--quant-embd']' returned non-zero exit status 1., check details in logs/convert_to_tl.log

Checked log #cat logs/convert_to_tl.log INFO:hf-to-gguf:Loading model: BitNet-b1.58-2B-4T Traceback (most recent call last): File ".../BitNet/utils/convert-hf-to-gguf-bitnet.py", line 1165, in <module> main() File ".../BitNet/utils/convert-hf-to-gguf-bitnet.py", line 1140, in main hparams = Model.load_hparams(dir_model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../BitNet/utils/convert-hf-to-gguf-bitnet.py", line 222, in load_hparams with open(dir_model / "config.json", "r", encoding="utf-8") as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'models/BitNet-b1.58-2B-4T/config.json'

` ls -l models/BitNet-b1.58-2B-4T

total 2338840

-rw-r--r--@ 1 user staff 8959 Apr 24 21:02 README.md

-rw-r--r--@ 1 user staff 1187801280 Apr 24 21:03 ggml-model-i2_s.gguf `

[WanRui37]

Does using bitnet_b1_58-large help you?

bitnet_b1_58-large

[Lotfree618]

huggingface-cli download microsoft/BitNet-b1.58-2B-4T --local-dir models/BitNet-b1.58-2B-4T download "BitNet-b1.58-2B-4T", not "BitNet-b1.58-2B-4T-gguf"

[sd983527]

huggingface-cli download microsoft/BitNet-b1.58-2B-4T --local-dir models/BitNet-b1.58-2B-4T download "BitNet-b1.58-2B-4T", not "BitNet-b1.58-2B-4T-gguf"

this is not correct. the fp16 version is used for GPU inference. for CPU inference, please use the gguf version

[sd983527]

Following directions to setup on MacBook Pro M3

python3 setup_env.py -md models/BitNet-b1.58-2B-4T -q tl1 INFO:root:Compiling the code using CMake. INFO:root:Loading model from directory models/BitNet-b1.58-2B-4T. INFO:root:Converting HF model to GGUF format... ERROR:root:Error occurred while running command: Command '['<omitted>/BitNet/.venv/bin/python3', 'utils/convert-hf-to-gguf-bitnet.py', 'models/BitNet-b1.58-2B-4T', '--outtype', 'tl1', '--quant-embd']' returned non-zero exit status 1., check details in logs/convert_to_tl.log

Checked log #cat logs/convert_to_tl.log INFO:hf-to-gguf:Loading model: BitNet-b1.58-2B-4T Traceback (most recent call last): File ".../BitNet/utils/convert-hf-to-gguf-bitnet.py", line 1165, in <module> main() File ".../BitNet/utils/convert-hf-to-gguf-bitnet.py", line 1140, in main hparams = Model.load_hparams(dir_model) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".../BitNet/utils/convert-hf-to-gguf-bitnet.py", line 222, in load_hparams with open(dir_model / "config.json", "r", encoding="utf-8") as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'models/BitNet-b1.58-2B-4T/config.json'

` ls -l models/BitNet-b1.58-2B-4T

total 2338840

-rw-r--r--@ 1 user staff 8959 Apr 24 21:02 README.md

-rw-r--r--@ 1 user staff 1187801280 Apr 24 21:03 ggml-model-i2_s.gguf `

if you have built llama.cli, you can directly try to inference it.