Add ModernBERT to KerasHub

[gcuder]

ModernBERT has been released on HuggingFace. Since you can already load models from there into a Keras Backbone, it would be interesting to know, if this works as well.

https://huggingface.co/answerdotai/ModernBERT-base

[mattdangerw]

It does not yet work, as our Bert implementation does not support RoPE and alternating local/global attention.

Likely we would want to do this as a separate architecture (e.g. in a models/modern_bert/ directory), instead of sharing code with BERT, as its a substantially different architecture.

But we are planning to add it! It's a great fit for our library.

[gcuder]

Cool, let me know if I can help contributing to it.

[pranavvp16]

Hello is @sachinprasadhs still working on this I was thinking to pick this up and start working :)

[sachinprasadhs]

@pranavvp16 , Assigning it to you since no one has started working on it.

[pranavvp16]

Thanks I'll keep posting updates on this thread :)

[gcuder]

Found this resource and thought it could help https://github.com/sionic-ai/ModernBert-GTE-Model-Converter

[SauravMaheshkar]

Hi 👋🏼 , I'm coming at this from the KerasHub Synpase Project.

CC: @mattdangerw for clarity. Happy to let @pranavvp16 continue the work.

[divyashreepathihalli]

Hi Saurav! we haven't received any contribution here yet. Please feel free to open your PR.