When using torch.hub to do monocular estimation, it got RuntimeError

[LucioSunj]

When using torch.hub to do monocular estimation, it got RuntimeError:

  model = torch.hub.load('yvanyin/metric3d', 'metric3d_vit_giant2', pretrain=True)
  File "/home/qizhong/miniconda3/envs/gaussian_splatting/lib/python3.7/site-packages/torch/hub.py", line 540, in load
    model = _load_local(repo_or_dir, model, *args, **kwargs)
  File "/home/qizhong/miniconda3/envs/gaussian_splatting/lib/python3.7/site-packages/torch/hub.py", line 569, in _load_local
    model = entry(*args, **kwargs)
  File "/home/qizhong/.cache/torch/hub/yvanyin_metric3d_main/hubconf.py", line 135, in metric3d_vit_giant2
    model = get_configured_monodepth_model(cfg)
  File "/home/qizhong/.cache/torch/hub/yvanyin_metric3d_main/mono/model/monodepth_model.py", line 36, in get_configured_monodepth_model
    model = get_monodepth_model(cfg)
  File "/home/qizhong/.cache/torch/hub/yvanyin_metric3d_main/mono/model/monodepth_model.py", line 20, in get_monodepth_model
    model = DepthModel(cfg, **kwargs)
  File "/home/qizhong/.cache/torch/hub/yvanyin_metric3d_main/mono/model/monodepth_model.py", line 7, in __init__
    super(DepthModel, self).__init__(cfg)   
  File "/home/qizhong/.cache/torch/hub/yvanyin_metric3d_main/mono/model/model_pipelines/__base_model__.py", line 10, in __init__
    self.depth_model = get_func('mono.model.model_pipelines.' + model_type)(cfg)
  File "/home/qizhong/.cache/torch/hub/yvanyin_metric3d_main/mono/model/model_pipelines/dense_pipeline.py", line 9, in __init__
    self.encoder = get_func('mono.model.' + cfg.model.backbone.prefix + cfg.model.backbone.type)(**cfg.model.backbone)
  File "/home/qizhong/.cache/torch/hub/yvanyin_metric3d_main/mono/utils/comm.py", line 161, in get_func
    raise RuntimeError(f'Failed to find function: {func_name}')
RuntimeError: Failed to find function: mono.model.backbones.vit_giant2_reg

And my code is just like: model = torch.hub.load('yvanyin/metric3d', 'metric3d_vit_giant2', pretrain=True) pred_depth, confidence, output_dict = model.inference({'input': rgb}) pred_normal = output_dict['prediction_normal'][:, :3, :,:] # only available for Metric3Dv2 i.e., ViT models normal_confidence = output_dict['prediction_normal'][:, 3, :, :]

Could you please help me?

[Bearick]

Same problem here. Could @YvanYin provide some help?

[ahinchoi]

Hello, I encountered the same problem and fixed it. The problem was that for some unknown reason, mono.utils.comm.get_func() could not access the objects inside mono.model.backbones. As a temporary solution, inside mono/tools/test_scale_cano.py, I simply added from mono.model.backbones import * . That solved the issue for me. Best regards!

[JUGGHM]

I am not quite sure about the torch.hub parts. Could Zach @ZachL1 introduce some experience here?

[ZachL1]

Hi, @LucioSunj @Bearick. Is this issue still occurring? Everything is working fine on my end.

I can't reproduce your issue, and I can't deduce the possible cause from the information you've provided. Could you try the following steps and provide more information to help resolve this?

Navigate to the torchhub cache:

cd ~/.cache/torch/hub/yvanyin_metric3d_main

Start the Python environment and try importing the package:

from mono.model.backbones import vit_giant2_reg

If the import fails, please check the mono/model/backbones/__init__.py file and the mono/model/backbones/ directory.

[yifanlu0227]

It also occurs for me, but from time to time.

I add from mono.model.backbones import * in ~/.cache/torch/hub/yvanyin_metric3d_main/mono/utils/comm.py and the problem get fixed. I guess something buggy with importlib

[ZachL1]

@yifanlu0227 Thanks for your comment, this can be a solution when meeting this problem.

I did find some discussions about importlib in the torchhub community, but I am unable to determine if they are related to this issue. So far, the discussion doesn't have comments that can infer the root cause, and I cannot reproduce the problem.

I provided a demo in Colab for loading Metric3D via TorchHub and performing inference, and I ran it 30 times with everything working well. See: Colab Link. If you can provide an environment to reproduce this issue, whether in Colab or Docker, it would maybe help resolve this problem.

[Zoe-Wan]

Can we finetune this model by torch.hub? I want to fine tune it on nuScenes Dataset.

[KrOverdulve]

In my case the issue was not having installed timm. It was resolved by doing

pip install timm