ninja: build stopped: subcommand failed.

Open utsunai opened this issue 2 years ago • 1 comments
I met this error, Plz help if you know what's going on thank you!
Traceback (most recent call last):
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1509, in _run_ninja_build
    subprocess.run(
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "demo.py", line 78, in <module>
    run_hrn(args)
  File "demo.py", line 26, in run_hrn
    output = reconstructor.predict(img, visualize=True, save_name=save_name, out_dir=out_dir)
  File "/home/tianhy/HRN/models/hrn.py", line 238, in predict
    output = self.predict_base(img)
  File "/home/tianhy/HRN/models/hrn.py", line 165, in predict_base
    output = self.model.predict_results_base()  # run inference
  File "/home/tianhy/HRN/models/facerecon_model.py", line 278, in predict_results_base
    pred_mask, _, pred_face = self.renderer.render_uv_texture(face_vertex, self.facemodel_front.face_buf,
  File "/home/tianhy/HRN/util/nv_diffrast.py", line 173, in render_uv_texture
    self.glctx = dr.RasterizeGLContext(device=device)
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/torch/ops.py", line 150, in __init__
    self.cpp_wrapper = _get_plugin().RasterizeGLStateWrapper(output_db, mode == 'automatic', cuda_device_idx)
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/torch/ops.py", line 83, in _get_plugin
    torch.utils.cpp_extension.load(name=plugin_name, sources=source_paths, extra_cflags=opts, extra_cuda_cflags=opts, extra_ldflags=ldflags, with_cuda=True, verbose=False)
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 963, in load
    return _jit_compile(
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1170, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1276, in _write_ninja_file_and_build_library
    _run_ninja_build(
  File "/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1529, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error building extension 'nvdiffrast_plugin': [1/5] /home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/rasterize.cu -o rasterize.cuda.o
FAILED: rasterize.cuda.o
/home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/rasterize.cu -o rasterize.cuda.o
/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/rasterize.cu(19): warning #177-D: variable "_ca_temp" was declared but never referenced
      float _ca_temp;
            ^

Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"

ptxas /tmp/tmpxft_0000333a_00000000-6_rasterize.ptx, line 9; fatal   : Unsupported .version 8.3; current version is '7.7'
ptxas fatal   : Ptx assembly aborted due to errors
[2/5] /home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/interpolate.cu -o interpolate.cuda.o
FAILED: interpolate.cuda.o
/home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/interpolate.cu -o interpolate.cuda.o
/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/interpolate.cu(135): warning #177-D: variable "_ca_temp" was declared but never referenced
      float _ca_temp;
            ^

Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"

ptxas /tmp/tmpxft_0000333c_00000000-6_interpolate.ptx, line 9; fatal   : Unsupported .version 8.3; current version is '7.7'
ptxas fatal   : Ptx assembly aborted due to errors
[3/5] /home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/antialias.cu -o antialias.cuda.o
FAILED: antialias.cuda.o
/home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/antialias.cu -o antialias.cuda.o
/home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/antialias.cu(390): warning #177-D: variable "_ca_temp" was declared but never referenced
      float _ca_temp;
            ^

Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"

ptxas /tmp/tmpxft_0000333e_00000000-6_antialias.ptx, line 9; fatal   : Unsupported .version 8.3; current version is '7.7'
ptxas fatal   : Ptx assembly aborted due to errors
[4/5] /home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/texture.cu -o texture.cuda.o
FAILED: texture.cuda.o
/home/tianhy/miniconda3/envs/HRN/bin/nvcc -DTORCH_EXTENSION_NAME=nvdiffrast_plugin -DTORCH_API_INCLUDE_EXTENSION_H -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/TH -isystem /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/torch/include/THC -isystem /home/tianhy/miniconda3/envs/HRN/include -isystem /home/tianhy/miniconda3/envs/HRN/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -DNVDR_TORCH -std=c++14 -c /home/tianhy/miniconda3/envs/HRN/lib/python3.8/site-packages/nvdiffrast/common/texture.cu -o texture.cuda.o
ptxas /tmp/tmpxft_0000333d_00000000-6_texture.ptx, line 9; fatal   : Unsupported .version 8.3; current version is '7.7'
ptxas fatal   : Ptx assembly aborted due to errors
ninja: build stopped: subcommand failed.
Nov 25 '23 13:11 utsunai
We haven't had this problem before. Maybe you can try to change all the "dr.RasterizeGLContext" in util/nv_diffrast.py into "dr.RasterizeCudaContext".
Jan 05 '24 07:01 youngLBW