ocannl
ocannl copied to clipboard
Add example: a decoder-only autoregressive Transformer language model
It is out-of-scope for this issue to implement support for open-weights models such as GPT-2 or Gemma: I'm leaving that for version 0.7. We should avoid doing work on a tokenizer here -- for version 0.7 we will use upcoming bindings to huggingface-tokenizers . Therefore, this can really be a toy example, maybe with the names dataset?