[WIP] Add H3

[NielsRogge]

What does this PR do?

This PR adds the H3 model by Hazy Research (Stanford University).

I've removed the Flash Attention dependency, and main author @DanFu09 has removed the einops dependency (🙏 ).

I've kept an optional soft dependency on pykeops, to allow for speedups. The model runs fine if the user doesn't have this library installed.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[gaceladri]

Hey @NielsRogge ✋,

Nice work! Do you know if this model would be integrated in the near future inside HuggingFace? Was this PR staled given complexities with custom ops?

Could you give an overview of the missing steps needed in this PR to have a functional H3 model integrated into HF? 🙏 Thanks for your work! 🙌

[NielsRogge]

Hi @gaceladri the PR is actually totally ready, the only thing that needs to done is perhaps make this function more like the other Attention classes in the library (like this class).

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.