nix-apollo issues

Repositories
Issues
Comments

Results 1 issues of


                                            nix-apollo

[Bug Report] Tiny stories models have longer n_ctx than they were trained with

**Describe the bug** The huggingface tiny stories configs claim to support `n_ctx=2048`. However, the model was only trained with sequence length 512 (as mentioned [here](https://huggingface.co/roneneldan/TinyStories-33M)). The models in fact get...