Yuqian Yuan

Hi, @prabin333 you should download [ViT-B SAM model](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth) to `./checkpoints`, or change this line to the path of ViT-B SAM model. https://github.com/CircleRadon/Osprey/blob/ce786ba5f612de3166039c4a9c196f77352f702c/demo/inference.py#L8

It appears that the path to the CLIP model is incorrect. Based on the error message and the location of your checkpoints, please update [this line](https://huggingface.co/sunshine-lwt/Osprey-7b/blob/50b6ac5e27319ee93ce60c9d33ae1f32d17423bc/config.json#L19) in the config.json file...

In colab, localhost is not accessible, you can set `os.environ["no_proxy"] = "localhost,127.0.0.1,::1”` in colab. I tried it and it works. 

Hi, @zwyang6 It seems that you didn't compile the op or didn't compile the op successfully. ```shell cd apro/gp_cuda python setup.py build develop ```

@kuaileqipaoshui Hi, we crop the bounding box of the mask and use it as input of LLaVA, as shown in the paper.