rustformers
GGML README
Rendered document can be viewed here https://github.com/danforbes/llama-rs/blob/dfo/doc/ggml/crates/ggml/README.md
I need help with (at least) vocabulary-scoring before this is ready to come out of draft.
This is really well done! Awesome work, and sorry about not looking at it earlier!
Honestly, it's so thorough I'm wondering if I should encourage you to submit it to upstream GGML instead. They could certainly benefit from the documentation!
Can you write one sentence that describes the purpose of token-scoring? Is this correct?
Depending on the GGML version, the token may also include a 32-bit floating point score, which impacts how likely that token is to appear in text generated by the model.
Can you write one sentence that describes the purpose of token-scoring? Is this correct?
Depending on the GGML version, the token may also include a 32-bit floating point score, which impacts how likely that token is to appear in text generated by the model.
I'd say more "how likely that token is to appear in the dataset that the model was trained on", but otherwise yeah that's fine
Honestly, it's so thorough I'm wondering if I should encourage you to submit it to upstream GGML instead. They could certainly benefit from the documentation!
Oh, that was always part of the plan 😉 https://github.com/ggerganov/ggml/pull/177