Phil Wang

if doing latent diffusion training, also worth thinking about whether to pre-encode the codebook ids and then select the codes from the codebook during training

good idea! yea we should definitely compile a set of prompts sorted by difficulty, open sourced in some repository then it would be trivial to preencode them for eval across...

Any progress?

@zhaobingbingbing you can add as many unets as you like in the cascade

@Birch-san haha yea, it is on my radar has anyone in the community / discord tried it yet? wondering how well it works

if @marunine vouches for the approach (or Katherine for that matter), I will definitely add it

> I'm sitting on a bit of data now waiting to test along these very lines. There is nothing in my mind that should fundamentally require T5 (would love to...

> Sub-quadratic attention could work in Imagen we already have this! https://github.com/lucidrains/imagen-pytorch/blob/main/imagen_pytorch/imagen_pytorch.py#L928 just set `use_linear_attn` and `use_linear_cross_attn` to `True`

@olegklimov Hi Oleg, you should try the open sourced softmax and blocksparse matmul by the author of Triton https://github.com/openai/triton/tree/master/python/triton/ops/blocksparse I will probably only focus on a simplified variant aimed to...

hmm, that looks like an error with kornia, a separate library