Mark

Bidirectional truncation in Llama4.

4

Really simple, just add argument that is already supported.

Fix GRPO recipe

1

#### Context What is the purpose of this PR? Is it to - [ ] add a new feature - [X] fix a bug - [ ] update tests and/or...

[DO NOT MERGE] Debug CI

1

RL CI is working improperly.

[WIP] Proper tool calling support in the torchtune

6

#### Context What is the purpose of this PR? Is it to - [X] add a new feature - [ ] fix a bug - [ ] update tests and/or...

Qwen3-235B-A22B OOMs with sufficient amount of VRAM

4

We're encountering an out-of-memory (OOM) error on a 2TB machine when attempting to load `Qwen3-235B-A22B`. The issue seems to be that `torchtune` loads the full, unsharded checkpoint with `self._checkpoint_client.load_base_checkpoint()`. For...

Reimplement `batched_rewards` to fix compatibility, broken in #2698

3

#### Context What is the purpose of this PR? Is it to - [ ] add a new feature - [X] fix a bug - [ ] update tests and/or...

Reference-free DPO losses in torchtune.

5

#### Context What is the purpose of this PR? Is it to - [X] add a new feature - [ ] fix a bug - [X] update tests and/or documentation...

[RFC] Reward modeling

1

Reward modeling in torchtune RFC Core issues We do not have an out-of-the-box toolkit to perform state-of-the-art reward modeling in torchtune; While the standard reward models are usually trained by...

Fix command in config

6

#### Context What is the purpose of this PR? Is it to - [ ] add a new feature - [x] fix a bug - [ ] update tests and/or...

Option to clip logprobs `rlhf.get_batch_log_probs`

7

RLHF procedures with modern DPO functionals may lead to the degenerate solution, e.g., the EOS token dropping during the generation. For instance, let's consider the output of the Qwen2.5 model...