transformers

transformers copied to clipboard

Differential Attention implementation for BERT.

1

### Feature request The paper "Differential Transformers" implements a differential attention mechanism which calculates the attention scores as the difference between two separate softmax attention maps leading to better long-context...

Chirayu-Tripathi

Retain newlines in chat template when `continue_final_message=True`

1

# What does this PR do? This PR addresses an issue with chat templates producing incorrect inputs when `continue_final_message=True` and we wish to terminate generation on newlines like `\n` (as...

lewtun

The method `model_download_tool` was called `model_download_counter` earlier in the tutorial, this raises an error when following the code. # What does this PR do? This PR fixes a small typo...

andimarafioti

Image-Text-to-Text Support in Transformers Pipeline

2

### Feature request Implement the new feature to support a pipeline that can take both an image and text as inputs, and produce a text output. This would be particularly...

chakravarthik27

Fix continue_final_message for image-text-to-text chat templates

1

# What does this PR do? The `content` field for an image-text-to-text model is a list, which is not currently taken into account when `continue_final_message` is set to True in...

yonigozlan

# What does this PR do? Currently, the modeling file of Paligemma uses `from ...utils import logging`, and the processing file uses `import logging`. That means if we want to...

yonigozlan

### Model description Janus is an autoregressive framework that unifies multimodal understanding and generation. Unlike previous approaches that use a single visual encoder for both tasks, Janus decouples visual encoding...

ighoshsubho

Add DDP token averaging for equivalent non-parallel training similar to #34191

1

### Feature request Token averaging in gradient accumulation was fixed in #34191 . But token averaging in DDP seems to have the same issue. --- ## Expected behaivor With all...

sbwww

Add image text to text pipeline

6

# What does this PR do? Add image-text-to-text pipeline! A split of this PR with only model-specific pre and post processing is available [here](https://github.com/huggingface/transformers/pull/32471), in order to reduce the loc...

yonigozlan

MLlama does not work with FSDP mix-precision

3

### System Info transformers==4.45.2 ### Who can help? @ArthurZucker ### Information - [ ] The official example scripts - [X] My own modified scripts ### Tasks - [ ] An...

YangFei1990