DepthFlow icon indicating copy to clipboard operation
DepthFlow copied to clipboard

Enhancement:Integrate Apple's open-source ml-depth-pro depth model

Open dedh121224 opened this issue 1 year ago • 3 comments

🔘 Request type

Enhancement

🔘 Description

Integrate Apple's open-source ml-depth-pro depth model

We propose integrating the ml-depth-pro depth model, recently open-sourced by Apple (https://github.com/apple/ml-depth-pro), into the project.

dedh121224 avatar Oct 06 '24 01:10 dedh121224

I saw it yesterday right before sleeping, will take a look today!

A side note: Sharpness isn't great for depthflow as it would cause artifacts on the edges. The most important thing is good relative depth values, we'll see how it goes :eyes:

Tremeschin avatar Oct 06 '24 12:10 Tremeschin

Implemented in https://github.com/BrokenSource/BrokenSource/commit/d898435ebc7e023048bb8458f795f51219a869cd, https://github.com/BrokenSource/DepthFlow/commit/c41694e285b1625deb4722f5d0f4dbc6d8c5c174

Note: currently only availabe when running from source until a new release!

image

Honestly, not impressed with the results 😓, maybe I'm doing something wrong but I followed their examples

Tremeschin avatar Oct 06 '24 14:10 Tremeschin

Just saying, but they don't provide a PyPI package, so I did a git+ install, it might break at any moment and without notice.

Tremeschin avatar Oct 06 '24 14:10 Tremeschin

I saw it yesterday right before sleeping, will take a look today!

A side note: Sharpness isn't great for depthflow as it would cause artifacts on the edges. The most important thing is good relative depth values, we'll see how it goes 👀

Just curious, for depth anything v2, would it be better to add a slight blur on the depth map to reduce artifacts on edges?

adi-panda avatar Oct 08 '24 22:10 adi-panda

@adi-panda Already doing it here! 🙂

Tremeschin avatar Oct 08 '24 22:10 Tremeschin

@adi-panda Already doing it here! 🙂

Ohh ok nice, amazing stuff!

adi-panda avatar Oct 08 '24 22:10 adi-panda

Version v0.8 released with DepthPro integration!

Tremeschin avatar Oct 27 '24 19:10 Tremeschin