ggml icon indicating copy to clipboard operation
ggml copied to clipboard

GGUF file format specification

Open philpax opened this issue 1 year ago • 28 comments

Closes #220.

Rendered: https://github.com/philpax/ggml/blob/gguf-spec/docs/gguf.md

Defines a complete specification for the proposed GGUF file format, which should generically describe models to be loaded by any compatible executor.

This is a first draft, so there's still some work that needs to be done - I need to fill in the TODOs and clarify a few things. If you have any suggestions for what should go in the TODOs, please let me know!

Changes from the version in the issue include:

  • changing of several of the key-value pairs, including splitting them out into per-architecture key-values
  • decoupling tensor info from tensor data, and aligning both
  • moving the embedded vocabulary into the metadata, so that it is no longer special-cased

philpax avatar Jun 25 '23 22:06 philpax