With SD (3.5 and XL) and MacOS, performance got much worse from latest release v0.2.6 to current master

[Itox001]

Expected Behavior

When I pull master, I get the same or better performance as with the revision that I had before.

Actual Behavior

I used to be able to run SD3.5 at around 7 seconds per iteration, but when I pulled master which is currently on revision fabf449febfc71f863f48052d83bfed7b1727a45 (Mochi VAE encoder.), the performance got worse, up to 11.5 seconds per iteration. I rolled back ComfyUI to the latest release version (0.2.6), and the performance was back to 7s per iteration.

Steps to Reproduce

Check out version v2.6.0, and run SD3.5. Check out revision fabf449febfc71f863f48052d83bfed7b1727a45, and run SD3.5. Performance should be considerably different.

Debug Logs

Running MacOS 15 on Apple silicon
Pytorch 2.6.0.dev-20241101
Workflow is the first example from https://comfyanonymous.github.io/ComfyUI_examples/sd3/, but with t5xxl_fp8_e4m3fn.safetensors and sd3.5_medium.safetensors

Other

No response