truly opensource model called olmo

[olumolu]

Build with truly open dataset and fully open-source model can this be supported in olllama thanks. https://allenai.org/olmo https://huggingface.co/allenai/OLMo-7B

[Jeximo]

There's a PR to implement this: https://github.com/ggerganov/llama.cpp/pull/6741

[schmorp]

While OLMo support has been merged, it doesn't work for any of the olmo models I tried:

Loading model: OLMo-7B-Twin-2T-hf
gguf: This GGUF file is for Little Endian only
Set model parameters
gguf: context length = 2048
gguf: embedding length = 4096
gguf: feed forward length = 11008
gguf: head count = 32
gguf: key-value head count = 32
gguf: rope theta = 10000.0
gguf: file type = 1
Traceback (most recent call last):
  File "llama.cpp/convert-hf-to-gguf.py", line 2807, in <module>
    main()
  File "llama.cpp/convert-hf-to-gguf.py", line 2791, in main
    model_instance.set_gguf_parameters()
  File "llama.cpp/convert-hf-to-gguf.py", line 2667, in set_gguf_parameters
    self.gguf_writer.add_clamp_kqv(self.hparams["clip_qkv"])
  File "llama.cpp/gguf-py/gguf/gguf_writer.py", line 369, in add_clamp_kqv
    self.add_float32(Keys.Attention.CLAMP_KQV.format(arch=self.arch), value)
  File "llama.cpp/gguf-py/gguf/gguf_writer.py", line 132, in add_float32
    self.add_val(val, GGUFValueType.FLOAT32)
  File "llama.cpp/gguf-py/gguf/gguf_writer.py", line 173, in add_val
    self.kv_data += self._pack(pack_fmt, val, skip_pack_prefix = vtype == GGUFValueType.BOOL)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "llama.cpp/gguf-py/gguf/gguf_writer.py", line 515, in _pack
    return struct.pack(f'{pack_prefix}{fmt}', value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: required argument is not a float
job finished, status 1

[schmorp]

OLMo-7B-SFT fails differently:

KeyError: "could not find any of: ['hidden_size', 'n_embd']"

[nopperl]

@schmorp I've fixed the conversion bug in #6910.

OLMo-7B-SFT fails differently:

KeyError: "could not find any of: ['hidden_size', 'n_embd']"

This is because the conversion script only supports OLMo in the format supported natively by HF transformers (i.e., all models with the -hf suffix here). The OLMo-7B-SFT hasn't been converted to the HF format yet.

Also, make sure to use transformers>=4.40.0.