schmorp

@JohannesGaessler I don't doubt (and have not doubted) your good faith. All my model cards have essentially the same text with variations (e.g. https://huggingface.co/mradermacher/Meta-Llama-3-70B-GGUF). The text said "weighted/imatrix quants of...

PS: the repository no longer exists because I delete repos that are demonstrated to be broken. I have done this numerous times, and if the OR would actually have discussed...

Oh, and an even better example is https://huggingface.co/mradermacher/llama-3-70B-instruct-uncensored-i1-GGUF where I documented that llama.cpp crashes on some quants, and the metadata documents the crash reason: `no_imatrix: 'GGML_ASSERT: llama.cpp/ggml-quants.c:11239: grid_index >= 0'`...

@JohannesGaessler I have never seen imatrix recover from nans just by using more tokens. In fact, fewer tokens have a higher chance of not running into the problem in the...

@slaren thanks for the clarification. however, you made a statement of fact, and I have to go by what you actually write, so don't make this my fault somehow :)...

@slaren: I think the confusion here is between the source model and the quants (which are also models). Currently, both are being refused. Also, I simply think it goes a...

@JohannesGaessler so how would a sum of something + nans turn out into something not a nan, something you claimed would happen? Nobody has seen this happen, and numerically it...

While OLMo support has been merged, it doesn't work for any of the olmo models I tried: ``` Loading model: OLMo-7B-Twin-2T-hf gguf: This GGUF file is for Little Endian only...

OLMo-7B-SFT fails differently: KeyError: "could not find any of: ['hidden_size', 'n_embd']"

Pretty much a backdoor giving full access to all usder data. casaos is completely insecure by default, with no indication of being so to normal users. The fact that I...