huggingface
[Pipeline] Marigold depth and normals estimation
What does this PR do?
Fixes #7522. Related to #6533 (community pipeline).
This PR adds the family of Marigold pipelines for dense regression tasks. The depth estimation pipeline is largely an improved version of the community pipeline. The new normals estimation pipeline adds a new functionality of computing surface normals.
Before submitting
- [x] Did you read the contributor guideline?
- [x] Did you read our philosophy doc (important for complex PRs)?
- [x] Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case. #7522 #6533
- [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings. Work in progress.
- [ ] Did you write any new necessary tests? Work in progress.
Who can review?
@sayakpaul @yiyixuxu
Hi @sayakpaul, @yiyixuxu , I made a round of changes to address your comments. Thanks to your input, the pipelines look more like the other official pipelines. Please have a look. I will now continue with tests and docs
@BenjaminBossan can you give this a review too? this is a very cool diffusion-based depth estimation model!
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
i tested out a little bit here and made some comments https://github.com/huggingface/diffusers/pull/8064 main feedback is:
- we should add more fast tests (I added one basic ones, but feel free to add more)
- we should make sure np output is as expected (channel-last)
there is a CI failure on PR documentation that's not relevant to this PR
The documentation currently pulls files from the unmerged PRs here: https://huggingface.co/datasets/huggingface/documentation-images/discussions/338 https://huggingface.co/datasets/huggingface/documentation-images/discussions/339 https://huggingface.co/datasets/huggingface/documentation-images/discussions/340
Once these are merged, paths to images will need to be updated again
Hi @sayakpaul , I made a round of changes, please let me know what you think about the new separation of the documentation!
Thanks a mile for landing this one with such poise and greatness, @toshas!
Thanks, @sayakpaul and @yiyixuxu, for guiding me on this one, It was an amazing experience, and I am very happy with the result!