ocannl
ocannl copied to clipboard
Implement a BERT model, replicate ModernBERT to use ModernBERT weights
It's from Answer.AI (Jeremy Howard's new company). Sources:
- https://x.com/jeremyphoward/status/1869786023963832509
- https://huggingface.co/blog/modernbert
- https://arxiv.org/abs/2412.13663