test-text-llm-encodings-without-attention-layers-with-cerebros

[david-thrower]

Kind of issue: enhancement

Additional context: Another attempt to make a lighter weight equally robust model.

Suggested Labels (If you don't know, that's ok): kind/enhancement kind/performance

[david-thrower]

This is a strong candidate for merging in. The last thing left is to extend the seq length a little on another branch just to see if we can exceed 95% test set accuracy without too much computational work and still being under 50M params.