Support `Longformer` as a `model_type` in `RepresentationModel`

[mgh1]

Is your feature request related to a problem? Please describe. Longformer is a popular model type but not supported in RepresentationModel , only roberta, bert, and gp2 are supported. Please support longformer because it is becoming more and more popular as it is being found to be have SOTA performance on numerous tasks. Extracting the embedding would therefore be extremely useful.

Describe the solution you'd like Add longformer as a model_type in RepresentationModel

Describe alternatives you've considered Tried using the existing model_types but they produce poor results for the embedding since they are architecturally different.

Additional context N/A

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.