Junhui He

> I used my own dataset to finetune the model `bitnet-b1.58-2B-4T-bf16` for downstream task. The saved checkpoint directory is as follows: > > path/to/my/ckpt > ├── chat_template.jinja > ├── config.json...

> A few days ago i could run the inference as normal. Today is just doesn't work anymore. > > I downloaded the model and then setup according to the...

The solution is to use clang-19 as the compiler. Run: ``` export CC="clang-19" export CXX="clang++-19" ``` Then try to build the project again: ``` python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s...

`clang-19` is still not activated, and the current compilation is still using `gcc-13`. Please try this command: ``` cmake -B build -DGGML_BITNET_ARM_TL1=ON -DCMAKE_C_COMPILER=clang-19 -DCMAKE_CXX_COMPILER=clang++-19 ``` Let me know if you...

We implemented a standalone script for converting HF models to GGUF. Please refer to [https://github.com/microsoft/BitNet?tab=readme-ov-file#convert-from-safetensors-checkpoints](https://github.com/microsoft/BitNet?tab=readme-ov-file#convert-from-safetensors-checkpoints). If you do not need to finetune the pretrained model, we recommend downloading microsoft/BitNet-b1.58-2B-4T-gguf instead...

The conversion script is now ready. You can find the instructions on how to use it here: https://github.com/microsoft/BitNet/?tab=readme-ov-file#convert-from-safetensors-checkpoints Let us know if you encounter any issues or have further questions....

The difference in metadata like `general.architecture` and `general.name` is due to different versions of the `gguf` library. We used a customized internal version of the library to produce the official...

Did you miss the `--chat-format` parameter?

Actually, both `--chat-format` (with a hyphen) and `--chat_format` (with an underscore) are valid.