Ethan Smith

Is there or will there be support for xformers?

1

### Feature request xformers, (i couldn't find anything online or in the docs, but i suspect its very likely I'm just missing something) ### Motivation speed and memory improvement ###...

Buckets of elements of different sizes

4

I am attempting to use webdataset to support loading a dataset with subsections/buckets organized by image size. To do this, I've organized the files such that each bucket has its...

Question about original google implementation with stable diffusion

3

Hi bloc, firstly thank you for your great work! I've been spending a lot of time trying to implement google's original release into a custom pipeline with diffusers. I figured...

[BUG] assert all_groups_norm > 0 | Error related to Bf16 optimizer it seems

9

**Describe the bug** A clear and concise description of what the bug is. Many of my trainings when in the config we have: bf16=True, around a few hundred steps the...

**Describe the bug** When calling model_engine.save_checkpoint(), when bf16 is enabled frozen parameters will not be saved in the state_dict despite exclude_frozen_parameters=False by default. Upon disabling bf16 training behavior reverts to...

Tokenization slows towards end of dataset

3

### Describe the bug Mapped tokenization slows down substantially towards end of dataset. train set started off very slow, caught up to 20k then tapered off til the end. what's...

Unable to import ZoeDetector from the pip installation

2

I imagine the pip version is a version behind?

fix min-snr implementation

2

# What does this PR do? Fixes the implementation of min-snr training for v-prediction models Based on implementation seen here. https://github.com/kohya-ss/sd-scripts/blob/main/library/custom_train_functions.py#L66 These two graphs show the weight scheme based on...

[BUG] Concern around mixed precision training where weights are in low precision

2

I noticed that in deepspeed, when training with fp16 and bf16, weights are set to the lower precision. I am wondering if there is any chance of making this optional....