[Feature Request] Intergrate the new PatchFusion Depth Estimation Model into this project.

[J-Cott]

The Detail they have achieved looks very impressive, would it be possible to use this model in Automatic1111? : https://github.com/zhyever/PatchFusion

[semjon00]

Is it better than BOOST?

[graemeniedermayer]

Very Impressive! The hugging faces repo suggests it requires a large amount of VRAM, they suggest 24g. But the results look very good. This might be a more challenging integration.

[semjon00]

24g wow, that would be a beefy GPU! We went from something that could reasonably run on a CPU (smol Midas), then to average GPUs (BOOST or Zoedepth) and then into server hardware. Transformers might be a hot topic, but depth approximation is also growing rapidly and this beast is hungry for more flops! 😄

Ok, then who knows, maybe one day. If somebody can create a MR for this it would be a pleasure to merge. An integration would require refactoring is_boost boolean type of thing into patching enum kind of thing, with options [no, BOOST, PatchFusion] or something like this. The same for the UI.

[graemeniedermayer]

It looks like it uses midas, zoedepth, stable diffusion, and controlnet without offload vram though soo it should be possible to significantly reduce the requirement.

[semjon00]

Wait, it uses all these things at the same time? Interesting...