BitNet

BitNet copied to clipboard

Cannot quantize to f16 with i2_s

1

I am quite a newbie to LLM's, but was running into the following issue: Command: `python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s --quant-embd` Output: `ERROR:root:Error occurred while running command: Command '['/home/pis7/miniconda3/envs/bitnet-cpp/bin/python',...

pis7

Getting Errors When following Readme Instruction on ARM

1

In file included from /root/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-quants.h:4, from /root/BitNet/src/ggml-bitnet-lut.cpp:9: /root/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:154:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic] 154 | struct { | ^ /root/BitNet/3rdparty/llama.cpp/ggml/src/./ggml-common.h:175:16: warning: ISO C++ prohibits anonymous structs [-Wpedantic] 175...

zwx109473

## Description Error when running the llama-bench tool on a dummy model, as specified in readme. Failed assertion in ggml.c relating to tile number for parallel processing. ## Steps to...

MattyAB

I followed [Basic Usage section](https://github.com/microsoft/BitNet?tab=readme-ov-file#basic-usage) in documentation, and it appears that when sending very basic prompts for completion, the Model returns irrelevant completions/answers in the good scenario, and in worst...

zvigrinberg

Add a command to pull git submodule `3rdparty` repository before running pip install, so the pip install command will run with latest updates from submodule branch ( for cases in...

zvigrinberg

check for build requirements and abort build process if not satisfied

4

- python>=3.9 - cmake>=3.22 - clang>=18 on windows use the msvc-clang (>=17) fix https://github.com/microsoft/BitNet/issues/34

bmerkle

Cannot run tl1 model on iOS

3

Hi all, I have tested the ggml-model-tl1.gguf (Llama3-8B-1.58-100B-tokens) model on iOS (iPhone 15 Pro) using the llamaSwiftUI sample project. However, I encountered an EXC_BAD_ACCESS error at ggml_vec_dot_f32(). Can you confirm...

luionTW

Can you support reordering and vector embedding models？

tanyo520

How to compile a server.exe?

1

Functionality similar to LLaMA.cpp HTTP Server. ./llama-server

clilyn1234

Certain characters crash bitnet model inference?

6

I've been working on securing the user input, escaping invalid characters, however I've encountered a few prompts which cause the llama-cli to abruptly halt: ``` .\llama-cli.exe --model "..\..\..\models\Llama3-8B-1.58-100B-tokens\ggml-model-i2_s.gguf" --prompt "£"...

grctest