version unmatched when i build from source code

[ehuaa]

After I git clone this project, i tried to compile from source code. when i ran into bash ./scripts/build_pytorch_blade.sh i got this error while my pytorch version is

torch.version '2.1.0.dev20230316+cpu' and torchvision.version is '0.16.0.dev20230316+cpu'

which are the latest version right from the pip. Can you tell me a way not to downgrade my pytorch or torchvision and install BladeDisc Successfully by change build_pytorch_blade.sh or requirements.txt a little bit? Thank you very much!

[ehuaa]

@wyzero

[tanyokwok]

@ehuaa The issue was raised because the script wants to install torch==1.7.1+cu110 as it depends; this is configured via TORCH_BLADE_CI_BUILD_TORCH_VERSION, see https://github.com/alibaba/BladeDISC/blob/main/pytorch_blade/scripts/build_pytorch_blade.sh#L32.

BladeDISC already supports torch 2.0; You can skip the torch pip installation in the script build_pytorch_blade.sh.

[ehuaa]

@ehuaa The issue was raised because the script wants to install torch==1.7.1+cu110 as it depends; this is configured via TORCH_BLADE_CI_BUILD_TORCH_VERSION, see https://github.com/alibaba/BladeDISC/blob/main/pytorch_blade/scripts/build_pytorch_blade.sh#L32.

BladeDISC already supports torch 2.0; You can skip the torch pip installation in the script build_pytorch_blade.sh.

okay, thanks, i will try it later. And the docker image was not used during the installation of building from source code, i wonder if i missed some steps...

[ehuaa]

ERROR: @local_config_cuda//:enable_cuda :: Error loading option @local_config_cuda//:enable_cuda: no such package '@llvm-raw//utils/bazel': java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/8c712296fb75ff73db08f92444b35c438c01a405.tar.gz, https://github.com/llvm/llvm-project/archive/8c712296fb75ff73db08f92444b35c438c01a405.tar.gz] to /home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/external/llvm-raw/temp14641944661529926033/8c712296fb75ff73db08f92444b35c438c01a405.tar.gz: Premature EOF Traceback (most recent call last): File "/home/banach/BladeDISC/pytorch_blade/setup.py", line 151, in setup( File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/init.py", line 108, in setup return distutils.core.setup(**attrs) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command super().run_command(command) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/banach/BladeDISC/pytorch_blade/setup.py", line 107, in run self.cpp_run() File "/home/banach/BladeDISC/pytorch_blade/setup.py", line 91, in cpp_run build.test() File "/home/banach/BladeDISC/pytorch_blade/bazel_build.py", line 283, in test subprocess.check_call(test_cmd, shell=True, env=env, executable="/bin/bash") File "/home/banach/anaconda3/envs/p310/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -e; set -o pipefail; source .bazel_pyenv/bin/activate; bazel test --action_env PYTHON_BIN_PATH=/home/banach/anaconda3/envs/p310/bin/python3 --action_env BAZEL_LINKLIBS=-lstdc++ --action_env CC=/usr/bin/gcc --action_env CXX=/usr/bin/g++ --action_env DISC_FOREIGN_MAKE_JOBS=32 --copt=-DPYTORCH_VERSION_STRING="2.1.0.dev20230319+cu117" --copt=-DPYTORCH_MAJOR_VERSION=2 --copt=-DPYTORCH_MINOR_VERSION=1 --copt=-DTORCH_BLADE_CUDA_VERSION=11.7 --action_env TORCH_BLADE_TORCH_INSTALL_PATH=/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/torch --copt=-DPYBIND11_COMPILER_TYPE="_gcc" --copt=-DPYBIND11_STDLIB="_libstdcpp" --copt=-DPYBIND11_BUILD_ABI="_cxxabi1011" --config=torch_debug --config=torch_tensorrt --action_env TENSORRT_INSTALL_PATH=/usr/local/TensorRT/ --action_env NVCC=/usr/local/cuda/bin/nvcc --config=torch_enable_quantization --config=torch_cxx11abi_0 --config=torch_cuda //tests/mhlo/... //pytorch_blade:torch_blade_test_suite //tests/torch-disc-pdll/tests/... //tests/torchscript/...' returned non-zero exit status 2. I ran into this error after i comment the torch pip installation and the url above cannot be reached is there anything wrong with my installation steps? i didn't run the nvidia docker. @tanyokwok

[tanyokwok]

@ehuaa BladeDISC workspace is built with bazel. Thus we use bazel to resolve a lot of project third-party dependencies.

The error might be caused since there is a downloading failure. Please check your network and retry.

[ehuaa]

**/tests/torchscript:since_1_14.graph.test FAILED in 0.8s /home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/execroot/org_torch_blade/bazel-out/k8-dbg/testlogs/tests/torchscript/since_1_14.graph.test/test.log

Executed 37 out of 37 tests: 36 tests pass and 1 fails locally.** There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are. INFO: Build completed, 1 test FAILED, 12791 total actions Traceback (most recent call last): File "/home/banach/BladeDISC/pytorch_blade/setup.py", line 151, in setup( File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/init.py", line 108, in setup return distutils.core.setup(**attrs) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command super().run_command(command) File "/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/banach/BladeDISC/pytorch_blade/setup.py", line 107, in run self.cpp_run() File "/home/banach/BladeDISC/pytorch_blade/setup.py", line 91, in cpp_run build.test() File "/home/banach/BladeDISC/pytorch_blade/bazel_build.py", line 283, in test subprocess.check_call(test_cmd, shell=True, env=env, executable="/bin/bash") File "/home/banach/anaconda3/envs/p310/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'set -e; set -o pipefail; source .bazel_pyenv/bin/activate; bazel test --action_env PYTHON_BIN_PATH=/home/banach/anaconda3/envs/p310/bin/python3 --action_env BAZEL_LINKLIBS=-lstdc++ --action_env CC=/usr/bin/gcc --action_env CXX=/usr/bin/g++ --action_env DISC_FOREIGN_MAKE_JOBS=32 --copt=-DPYTORCH_VERSION_STRING="2.1.0.dev20230319+cu117" --copt=-DPYTORCH_MAJOR_VERSION=2 --copt=-DPYTORCH_MINOR_VERSION=1 --copt=-DTORCH_BLADE_CUDA_VERSION=11.7 --action_env TORCH_BLADE_TORCH_INSTALL_PATH=/home/banach/anaconda3/envs/p310/lib/python3.10/site-packages/torch --copt=-DPYBIND11_COMPILER_TYPE="_gcc" --copt=-DPYBIND11_STDLIB="_libstdcpp" --copt=-DPYBIND11_BUILD_ABI="_cxxabi1011" --config=torch_debug --config=torch_tensorrt --action_env TENSORRT_INSTALL_PATH=/usr/local/TensorRT/ --action_env NVCC=/usr/local/cuda/bin/nvcc --config=torch_enable_quantization --config=torch_cxx11abi_0 --config=torch_cuda //tests/mhlo/... //pytorch_blade:torch_blade_test_suite //tests/torch-disc-pdll/tests/... //tests/torchscript/...' returned non-zero exit status 3.

After i fixed the network problem, i failed with one test above ,is this the reason of the traceback above? the info in since_1_14.graph.test/test.log is:

TEST 'MLIR torchscript :: since_1_14.graph' FAILED Script:

: 'RUN: at line 1'; /home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/execroot/org_torch_blade/bazel-out/k8-dbg/bin/tests/torchscript/since_1_14.graph.test.runfiles/org_torch_blade/tests/torchscript/shape_analysis_tool --since 1.14.0 -f /home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/execroot/org_torch_blade/bazel-out/k8-dbg/bin/tests/torchscript/since_1_14.graph.test.runfiles/org_torch_blade/tests/torchscript/since_1_14.graph | /home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/execroot/org_torch_blade/bazel-out/k8-dbg/bin/tests/torchscript/since_1_14.graph.test.runfiles/llvm-project/llvm/FileCheck /home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/execroot/org_torch_blade/bazel-out/k8-dbg/bin/tests/torchscript/since_1_14.graph.test.runfiles/org_torch_blade/tests/torchscript/since_1_14.graph

Exit Code: 1

Command Output (stderr):

terminate called after throwing an instance of 'torch::jit::ErrorReport' what():
Couldn't find an operator for aten::var.correction(Tensor self, int[1]? dim, , int? correction, bool keepdim=False) -> Tensor. Do you have to update a set of hardcoded JIT ops?failed shape propagation in this context. The above operation: %4 : Tensor = aten::amax(%p1, %3, %1) The inputs are: Float(, *, *, ) from %p1 : Float(, *, *, *) = prim::Param() int[] from %3 : int[] = prim::ListConstruct(%2) bool from %1 : bool = prim::Constantvalue=1 : %cst_1: int = prim::Constantvalue=-1 %dims : int[] = prim::ListConstruct(%cst_1) %1 : Tensor = aten::amax(%p1, %dims, %true) ~~~~ <--- HERE return (%1)

/home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/execroot/org_torch_blade/bazel-out/k8-dbg/bin/tests/torchscript/since_1_14.graph.test.runfiles/org_torch_blade/tests/torchscript/since_1_14.graph:15:17: error: CHECK-LABEL: expected string not found in input // CHECK-LABEL: graph ^ :1:6: note: scanning from here graph(%self : Float(*, )): ^ :1:16: note: possible intended match here graph(%self : Float(, *)): ^

Input file: Check file: /home/banach/.cache/bazel/_bazel_banach/73d137a07d4a9c12dceaec8145974e25/execroot/org_torch_blade/bazel-out/k8-dbg/bin/tests/torchscript/since_1_14.graph.test.runfiles/org_torch_blade/tests/torchscript/since_1_14.graph

-dump-input=help explains the following input dump.

Input was: <<<<<< 1: graph(%self : Float(*, *)): label:15'0 X~~~~~~~~~~~~~~~~~~~~~~ error: no match found label:15'1 ? possible intended match 2: %1 : int = prim::Constantvalue=0 label:15'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 3: %2 : int = prim::Constantvalue=1 label:15'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4: %3 : int = prim::Constantvalue=32 label:15'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 5: %4 : int = prim::Constantvalue=512 label:15'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 6: %5 : int[] = prim::ListConstruct(%3, %4, %2) label:15'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . . .

Looking foward to your reply, thanks! @tanyokwok

[ehuaa]

i fixed this problem by pull the latest pr you committed last week, thanks!