Depth-Anything icon indicating copy to clipboard operation
Depth-Anything copied to clipboard

Add Hugging Face integration

Open NielsRogge opened this issue 5 months ago • 3 comments

Hi,

Thanks for this nice work. I wrote a quick PoC to showcase that you can easily have integration so that you can automatically load the various Depth Anything models using from_pretrained (and push them using push_to_hub), track download numbers for your models (similar to models in the Transformers library), and have nice model cards on a per-model basis. It leverages the PyTorchModelHubMixin class which allows to inherit these methods.

Usage is as follows:

import numpy as np
from PIL import Image
import torch

from depth_anything.dpt import DepthAnything
from depth_anything.util.transform import Resize, NormalizeImage, PrepareForNet
from torchvision.transforms import Compose

model = DepthAnything.from_pretrained("nielsr/depth_anything_dinov2_vits14")

transform = Compose([
        Resize(
            width=518,
            height=518,
            resize_target=False,
            keep_aspect_ratio=True,
            ensure_multiple_of=14,
            resize_method='lower_bound',
            image_interpolation_method=cv2.INTER_CUBIC,
        ),
        NormalizeImage(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
        PrepareForNet(),
    ])

img = Image.open(...)
image = transform({'image': np.array(image)})['image']
image = torch.from_numpy(image).unsqueeze(0)

depth = model(image)

The corresponding model is here for now: https://huggingface.co/nielsr/depth_anything_dinov2_vits14. We could move all checkpoints to separate repos on the TikToken organization if you're interested.

I saw that currently download numbers won't be tracked since all checkpoints are just part of a single repository.

Would you be interested in this integration?

Kind regards,

Niels

NielsRogge avatar Jan 22 '24 14:01 NielsRogge