BitNet
BitNet copied to clipboard
Cannot quantize to f16 with i2_s
I am quite a newbie to LLM's, but was running into the following issue:
Command: python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s --quant-embd
Output: ERROR:root:Error occurred while running command: Command '['/home/pis7/miniconda3/envs/bitnet-cpp/bin/python', 'utils/convert-hf-to-gguf-bitnet.py', 'models/Llama3-8B-1.58-100B-tokens', '--outtype', 'f32']' died with <Signals.SIGKILL: 9>., check details in logs/convert_to_f32_gguf.log
Specified log output does not show anything wrong (no errors).