X-InstructBLIP Installation failing

[giuliannocappellari]

I'm trying to install and run X-InstructBLIP, but I'm getting a issue.

I followed your instructions and I'm getting the following error when I executed load_model("blip2_vicuna_xinstruct", "vicuna7b"):

Traceback (most recent call last):
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 2100, in _run_ninja_build
    subprocess.run(
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/__init__.py", line 117, in load_model
    model = registry.get_model_class(name).from_pretrained(model_type=model_type)
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/base_model.py", line 70, in from_pretrained
    model = cls.from_config(model_cfg)
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/blip2_models/blip2_vicuna_xinstruct.py", line 2253, in from_config
    model = cls(
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/blip2_models/blip2_vicuna_xinstruct.py", line 259, in __init__
    modality_encoder, modality_ln = getattr(self, f"init_{modality}_encoder")(
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/blip2_models/blip2_vicuna_xinstruct.py", line 840, in init_pc_encoder
    pc_encoder = ULIP_PointBERT(ulip_v=2)
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/ulip_models/ULIP_models.py", line 196, in ULIP_PointBERT
    from lavis.models.ulip_models.pointbert.point_encoder import PointTransformer
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/ulip_models/pointbert/point_encoder.py", line 5, in <module>
    from lavis.models.ulip_models.pointbert.dvae import Group
  File "/opt/A/gabriel/LAVIS-XInstructBLIP/lavis/models/ulip_models/pointbert/dvae.py", line 7, in <module>
    from knn_cuda import KNN
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/knn_cuda/__init__.py", line 38, in <module>
    _knn = load_cpp_ext("knn")
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/knn_cuda/__init__.py", line 26, in load_cpp_ext
    ext = load(
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1308, in load
    return _jit_compile(
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1710, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1823, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 2116, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'knn': [1/2] /usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=knn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include/TH -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /opt/A/gabriel/conda_envs/lavis/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/knn_cuda/csrc/cuda/knn.cu -o knn.cuda.o 
FAILED: knn.cuda.o 
/usr/local/cuda/bin/nvcc  -DTORCH_EXTENSION_NAME=knn -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include/TH -isystem /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /opt/A/gabriel/conda_envs/lavis/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -std=c++17 -c /opt/A/gabriel/conda_envs/lavis/lib/python3.8/site-packages/knn_cuda/csrc/cuda/knn.cu -o knn.cuda.o 
nvcc fatal   : Value 'c++17' is not defined for option 'std'
ninja: build stopped: subcommand failed

[ljj898]

same problem, did u get fixed?

[artemisp]

Did you install ninja? wget -P /usr/bin https://github.com/unlimblue/KNN_CUDA/raw/master/ninja If that did not work, can you try: python -m pip install ninja