Easy-Transformer icon indicating copy to clipboard operation
Easy-Transformer copied to clipboard

[Proposal] support simplestories models

Open mivanit opened this issue 6 months ago • 0 comments

Proposal

Support the SimpleStories family of models.

Motivation

The TinyStories models already included in TransformerLens are incredibly useful both as objects of study in their own right, as well as for debugging research code in low-resource environments before investigating larger models. The SimpleStories models are an improvement on the TinyStories models, built on a much more diverse dataset.

Links

models on HF: https://huggingface.co/SimpleStories training repo: https://github.com/danbraunai/simple_stories_train dataset paper: https://arxiv.org/pdf/2504.09184

Checklist

  • [x] I have checked that there is no similar issue in the repo (required)

mivanit avatar Jun 17 '25 19:06 mivanit