Token merging

[lalalune]

Very interested in the work you're doing. Speed and memory efficiency are crucial for anyone trying to generate at scale.

We've implemented Token Merging: https://github.com/facebookresearch/ToMe/pull/7

This has a speed overhead increase of about 15% from the naive implementation at 512x512, although this goes up as array sizes increase. The memory overhead reduction is significant, and can allow for much larger image generation.