Kashif Rasul

[ORPO] add the log odds part to the loss

2

# What does this PR do? Adds the missing log-odds part to the loss

removed convnext from annotated diffusion post

1

# What does this PR do? Update the resnet block by removing convnext. Fixes # (issue) ## Who can review? Feel free to tag members/contributors who may be interested in...

some potential helpers

3

Feel free to delete this issue if not useful... ```python import torch def make_attention_mask(query_input: torch.Tensor, key_input: torch.Tensor, pairwise_fn=torch.multiply, extra_batch_dims=0, dtype=torch.bool): mask = pairwise_fn(query_input.unsqueeze(-1), key_input.unsqueeze(-2)) mask = mask.unsqueeze(-3) for dim in...

Instead of parameters with grad=False, buffers is the way to go.

[Wuerstchen] dreambooth lora

18

# What does this PR do? Lora dreambooth training scripts for Wuerstchen prior pipeline. Example: https://wandb.ai/krasul/dreambooth-lora/runs/7ia16ohd needs #6046

SOFTS from the paper: https://arxiv.org/pdf/2404.14197

SamFormer multivariate Estimator

3

Add samformer model https://arxiv.org/abs/2402.10198 cc @romilbert