Add TCD scheduler

[l0stl0rd]

Adds the TCD scheduler to better support. https://huggingface.co/h1t/TCD-SDXL-LoRA or checkpoints that have been made with TCD

Example: TCD Lora with Euler A

TCD Lora with TCD scheduler

[psychedelicious]

Thanks for the PR. I've tidied a couple things.

The scheduler works fine, but I'm not sure we fully support TCD at this time. I noticed a couple things when reviewing the TCD repo:

• The examples use a guidance of 0. This is unexpected. The app currently doesn't allow a value under 1. We can certainly change that if there's a reason to do so, but it's not clear if that exists.
• The examples add an eta arg to the diffusers pipeline and makes it sound like this should be user-configurable. It's not clear if this is really required or not.

I'm able to use the TCD SDXL LoRA, but it doesn't produce good results with the scheduler. I'm not sure if this is because I can only go as low as 1 for guidance, or the eta arg isn't provided, or if there is some other factor related to our diffusers implementation for the scheduler or LoRA.

Example output with the LoRA, 8 steps, CFG 1, per the examples:

(the higher the CFG, the worse the results)

Example output without the LoRA,

The scheduler does work fine without the TCD LoRA and with "normal" settings:

I'm reluctant to add the scheduler without the other changes to support the TCD model, because it communicates that we support hte TCD model (which appears to not untrue).

[l0stl0rd]

Thanks, it was the first time I tried adding something.

I noticed the same thing with the CFG for example, that is why I said on Discord, “it works but bot sure if it is correct.

I noticed the eta thing but was not sure what to do with it.

I was told in Invoke CFG 1 = 0 but not sure.

If you use the Lora at a strength 0,75 it is somewhat works but it should be usable at 1.

Also if it helps https://github.com/vladmandic/automatic support TCD and EDM (with diffusers) but for me as a non coder it was not very helpful.

[psychedelicious]

@l0stl0rd Sorry, I think I missed that discord thread. Can you link me to it? Can discuss the rest of the implementation there.

And thanks for the link to vlad's implementation, that will be a useful reference.

[l0stl0rd]

it was not in the dev chat but here it is https://discord.com/channels/1020123559063990373/1020123559831539744/1222968707240689694

You will see that I did not get EDM to work :(

[blessedcoolant]

The implementation in this PR is fine. The only issue I noticed is that when applying the LoRA, we're still getting a pretty noisy output at the default eta of 0.3 which is what Diffusers ships with. If we manually change that to 1.0, TCD is functioning as it should.

Lower values are okay too but from the little experimentation I did, it seems like 1.0 is what we set the gamma too for the step calculation.

I've updated this PR with that change. Please test and let me know. If 1.0 works fine in all cases and produces expected results, then we can keep it at that. Otherwise we'll have to expose this value for just that scheduler.

Here are the results of this model with Juggernaut X, TCD Scheduler and TCD SDXL LoRA at 4 steps and CFG Scale of 1. The right hand side image is the same settings + a Pixar Style LoRA to show that the TCD SDXL Lora is working perfectly fine with other LoRAs too.

When using TCD SDXL LoRA with a scheduler that is not TCD, you are recommended to use 8 steps instead of 4 for the best results.

[blessedcoolant]

This PR is fine to merge I think. Even the Hyper SDXL paper that utilizes the TCD Scheduler prefers the gamma value to be at 1.0 and as per my testing it holds up.

They released a new model today that allows guidance up to 5-7 with their new LoRA. That kinda fails for some reason but that is not related to this PR. I'll take a look at why that LoRA is not working correctly. The rest of the HyperSD LoRA's work just fine with the new TCD Scheduler.

[blessedcoolant]

Sorry was just curious about the eta check, didn't mean to block this

Na you're right. Initially I did the eta check coz I thought it was only on the TCD but then I realized there were other schedulers that used gamma variance too. So added in a specific check for TCD. And given it is a keyword argument, we don't really need to check if it exists or not. If it ever gets removed upstream for whatever reason, given it is a kwarg, it shouldn't break the app on our end. But having the double check might. So updated that.