transformers

transformers copied to clipboard

✨ Add EoMT Model || 🚨 Fix Mask2Former loss calculation

26

# What does this PR do? Fixes #37171 and continuation of #37392

yaswanth19

Improvements in attention_forward functions

23

# What does this PR do? Improves `eager_attention_forward` and `sdpa_attention_forward` in the case when `query.shape[1] > key.shape[1]`. This happens in GQA (grouped query attention), and in particular in multi-head latent...

mseeger

align xpu's autocast behavior w/ cuda by using device agnostic torch APIs

6

@ArthurZucker, pls help review, thx very much.

yao-matrix

feat: support indivisible shards for TP model loading and TPlizing.

7

# What does this PR do? Fixes https://github.com/huggingface/transformers/issues/37051 Approach is to support uneven sharding and seek segments of data that mimics torch.chunk since torch.chunk is the style of sharding adopted...

kmehant

Fix: Correctly handle integer device_map for NPU devices in _load_sta…

13

Fixes #38468 ### Problem Description This PR addresses the `AssertionError: "Torch not compiled with CUDA enabled"` that occurs when attempting to load models using `device_map="auto"` on systems with Ascend NPU...

gspeter-max

# What does this PR do? Draft PR to add Dust3R Cc @raf-fonseca

jadechoghari

`verify_tp_plan` function raises an error if a key without '.' is given

2

### System Info - `transformers` version: 4.53.0.dev0 - Platform: Linux-5.10.0-34-cloud-amd64-x86_64-with-glibc2.31 - Python version: 3.9.2 - Huggingface_hub version: 0.32.2 - Safetensors version: 0.4.5 - Accelerate version: 1.7.0 - Accelerate config: not...

liwii

Description: This pull request fixes minor typos in comments across two files: - In `modeling_swin.py`, the word "disible" was corrected to "divisible" for clarity in the padding comment. - In...

kilavvy

[WIP] Quartet QAT support

1

# This PR adds support for the Quartet QAT method. The goal of this PR is to integrate inference and training support for the [Quartet QAT method](https://arxiv.org/abs/2505.14669). That would allow...

BlackSamorez

`align_to_words=True` in `QuestionAnsweringPipeline` can lead to duplicate answers

17

### System Info - `transformers` version: 4.31.0 - Platform: macOS-13.4.1-arm64-arm-64bit - Python version: 3.11.4 - Huggingface_hub version: 0.15.1 - Safetensors version: 0.3.1 - Accelerate version: 0.21.0 - Accelerate config: not...

MichelBartels