Raushan Turganbay

Blip: get/set input embeddings correctly

3

# What does this PR do? Fixes https://github.com/huggingface/transformers/issues/34109 and adds `get_input_embeddings` method to the retrieval model. Also fixes the same methods in BLIP model where we should be working with...

VLM: add more modularity

1

# What does this PR do? As mentioned in https://github.com/huggingface/transformers/issues/33948, this PR simply refactors code a bit to make it more modular, Specifically we now will have special public methods...

# What does this PR do? Precedes https://github.com/huggingface/transformers/pull/34174/files. Before merging the linked PR, we need to stop relying on `main_input_name` as in some cases (BLIP) the main input is not...

VLMs: `patch_size` -> `num_image_tokens` in processing

9

# What does this PR do? Following https://github.com/huggingface/transformers/issues/33374, we'll use `num_image_tokens` instead of `patch_size` VLM in processors. The reason is because some image backbones add a CLS token while other...

# What does this PR do? Same as https://github.com/huggingface/transformers/pull/33754, I accidentally force pushed to wrong branch and the PR got closed 🙃

[internvl] fix video inference

4

# What does this PR do? Video inference broke after https://github.com/huggingface/transformers/pull/38105 because the model was inferring number of frames before processing. Previously that worked fine, but with the new workflow...

[configuration] remove redundant `classmethod`

2

# What does this PR do? As per title, another tiny clean up that deletes a `from_text_vision_configs` for initializing configs from its sub-configs. That can perfectly be done by `cls.__init__`...

Don't use cache in non-generative models

8

# What does this PR do? When working on https://github.com/huggingface/transformers/pull/38635, I found that there are some models which have `past_key_values` in their signature, even though they cannot generate. The reason...

[chat template] return assistant mask in processors

2

# What does this PR do? Fixes https://github.com/huggingface/transformers/issues/38521. I checked with fast tokenizers' implementation of `word_to_char` and saw no different in the time taken, so I think this can be...

Chemberta-2 tokenizer files incomplete

2

## 🐛 Bug The models for Chemberta2 on the HF hub are not complete. Their `merges.txt` file is empty which results in the SMILES ipnut being tokenized as a huge...