Bump transformers from 4.31.0 to 4.35.0

Sourced from transformers's releases.

Safetensors serialization by default, DistilWhisper, Fuyu, Kosmos-2, SeamlessM4T, Owl-v2

New models

Distil-Whisper

Distil-Whisper is a distilled version of Whisper that is 6 times faster, 49% smaller, and performs within 1% word error rate (WER) on out-of-distribution data. It was proposed in the paper Robust Knowledge Distillation via Large-Scale Pseudo Labelling.

Distil-Whisper copies the entire encoder from Whisper, meaning it retains Whisper's robustness to different audio conditions. It only copies 2 decoder layers, which significantly reduces the time taken to auto-regressively generate text tokens:

Distil-Whisper is MIT licensed and directly available in the Transformers library with chunked long-form inference, Flash Attention 2 support, and Speculative Decoding. For details on using the model, refer to the following instructions.

Joint work from @​sanchit-gandhi, @​patrickvonplaten and @​srush.

• [Assistant Generation] Improve Encoder Decoder by @​patrickvonplaten in #26701
• [WhisperForCausalLM] Add WhisperForCausalLM for speculative decoding by @​patrickvonplaten in #27195
• [Whisper, Bart, MBart] Add Flash Attention 2 by @​patrickvonplaten in #27203

Fuyu

The Fuyu model was created by ADEPT, and authored by Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, Sağnak Taşırlar.

The authors introduced Fuyu-8B, a decoder-only multimodal model based on the classic transformers architecture, with query and key normalization. A linear encoder is added to create multimodal embeddings from image inputs.

By treating image tokens like text tokens and using a special image-newline character, the model knows when an image line ends. Image positional embeddings are removed. This avoids the need for different training phases for various image resolutions. With 8 billion parameters and licensed under CC-BY-NC, Fuyu-8B is notable for its ability to handle both text and images, its impressive context size of 16K, and its overall performance.

Joint work from @​molbap, @​pcuenca, @​amyeroberts, @​ArthurZucker

• Add fuyu model by @​molbap in #26911
• Fuyu: improve image processing by @​molbap in #27007

SeamlessM4T

The SeamlessM4T model was proposed in SeamlessM4T — Massively Multilingual & Multimodal Machine Translation by the Seamless Communication team from Meta AI.

SeamlessM4T is a collection of models designed to provide high quality translation, allowing people from different linguistic communities to communicate effortlessly through speech and text.

SeamlessM4T enables multiple tasks without relying on separate models:

• Speech-to-speech translation (S2ST)
• Speech-to-text translation (S2TT)
• Text-to-speech translation (T2ST)
• Text-to-text translation (T2TT)
• Automatic speech recognition (ASR)

... (truncated)

• f1185a4 Release: v4.35.0
• b6c0c2b Fuyu protection
• fba863b Conda build
• 441c3e0 fix-deprecated-exllama-arg (#27243)
• 8801861 Fixing m4t. (#27240)
• 443bf5e Fix safetensors failing tests (#27231)
• 4557a0d Wrap _prepare_4d_causal_attention_mask as a leaf function (#27236)
• 8a31295 Fuyu: improve image processing (#27007)
• 9b25c16 [core / Quantization] Fix for 8bit serialization tests (#27234)
• c52e429 Reproducible checkpoint for npu (#27208)
• Additional commits viewable in compare view

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)