Kashif Rasul
Kashif Rasul
# What does this PR do? Adds the missing log-odds part to the loss
# What does this PR do? Update the resnet block by removing convnext. Fixes # (issue) ## Who can review? Feel free to tag members/contributors who may be interested in...
Feel free to delete this issue if not useful... ```python import torch def make_attention_mask(query_input: torch.Tensor, key_input: torch.Tensor, pairwise_fn=torch.multiply, extra_batch_dims=0, dtype=torch.bool): mask = pairwise_fn(query_input.unsqueeze(-1), key_input.unsqueeze(-2)) mask = mask.unsqueeze(-3) for dim in...
Instead of parameters with grad=False, buffers is the way to go.
# What does this PR do? Lora dreambooth training scripts for Wuerstchen prior pipeline. Example: https://wandb.ai/krasul/dreambooth-lora/runs/7ia16ohd needs #6046
SOFTS from the paper: https://arxiv.org/pdf/2404.14197
Add samformer model https://arxiv.org/abs/2402.10198 cc @romilbert