ocannl icon indicating copy to clipboard operation
ocannl copied to clipboard

Add example: a decoder-only autoregressive Transformer language model

Open lukstafi opened this issue 2 years ago • 0 comments

It is out-of-scope for this issue to implement support for open-weights models such as GPT-2 or Gemma: I'm leaving that for version 0.7. We should avoid doing work on a tokenizer here -- for version 0.7 we will use upcoming bindings to huggingface-tokenizers . Therefore, this can really be a toy example, maybe with the names dataset?

lukstafi avatar Feb 20 '23 10:02 lukstafi