Falcon port

[Rocketknight1]

This PR adds the Falcon model to the main library. It's still a work in progress, and integration tests / model checkpoints still need to be added!

TODO:

• [x] Migrate custom code checkpoints to the new architecture
• [x] Confirm tokenizer can be loaded correctly with AutoTokenizer for all checkpoints
• [x] Upload a ported 1B model.
• [x] Add integration tests for the 1B model
• [ ] Add support for output_attention and output_hidden_states
• [ ] Ensure all tests pass
• [ ] Ensure any other issues addressed (see comments on Slack)

[Rocketknight1]

Update: Slightly delayed because there are some breaking architecture changes between the different Falcon checkpoints - I'm merging the various layers and using config variables to switch between the behaviours.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ArthurZucker]

feel free to ping me for a review anytime!

[WilliamTambellini]

Hi @Rocketknight1 Would this PR allow to export falcon to onnx? As today using the latest release (4.30.1):

Traceback (most recent call last):
  File "hf2onnx.py", line 99, in <module>
    model_kind, model_onnx_config = FeaturesManager.check_supported_model_or_raise(model, feature= feature)
  File "site-packages/transformers/onnx/features.py", line 728, in check_supported_model_or_raise
    model_features = FeaturesManager.get_supported_features_for_model_type(model_type, model_name=model_name)
  File "site-packages/transformers/onnx/features.py", line 575, in get_supported_features_for_model_type
    raise KeyError(
KeyError: "refinedwebmodel is not supported yet. Only ['albert', 'bart', 'beit', 'bert', 'big-bird', 'bigbird-pegasus', 'blenderbot', 'blenderbot-small', 'bloom', 'camembert', 'clip', 'codegen', 'convbert', 'convnext', 'data2vec-text', 'data2vec-vision', 'deberta', 'deberta-v2', 'deit', 'detr', 'distilbert', 'electra', 'flaubert', 'gpt2', 'gptj', 'gpt-neo', 'groupvit', 'ibert', 'imagegpt', 'layoutlm', 'layoutlmv3', 'levit', 'longt5', 'longformer', 'marian', 'mbart', 'mobilebert', 'mobilenet-v1', 'mobilenet-v2', 'mobilevit', 'mt5', 'm2m-100', 'owlvit', 'perceiver', 'poolformer', 'rembert', 'resnet', 'roberta', 'roformer', 'segformer', 'squeezebert', 'swin', 't5', 'vision-encoder-decoder', 'vit', 'whisper', 'xlm', 'xlm-roberta', 'yolos'] are supported. If you want to support refinedwebmodel please propose a PR or open up an issue."

best

[Rocketknight1]

Hey all! The main modeling code should be ready for final review now. Thanks @ArthurZucker for the comprehensive review - it was really helpful! There's one bug left that's causing a failing test, but I think it's a one-line fix that I can track down tomorrow. This may also be the issue that's causing assisted generation to fail, but those tests are currently skipped.

I also need to figure out porting the tokenizer, and then once this is merged I'll need to prepare the repos to transition over to the library code.

cc @amyeroberts for core maintainer review!