Change ints to double in quant_cuda_kernel.cu?

[xiscoding]

I was getting this error when running python setup_cuda.py

quant_cuda_kernel.cu(149):` error: no instance of overloaded function "atomicAdd" matches the argument list argument types are: (double *, double) detected during instantiation of "void VecQuant2MatMulKernel(const scalar_t *, const int *, scalar_t *, const scalar_t *, const scalar_t *, int, int, int, int) [with scalar_t=double]" (87): here

quant_cuda_kernel.cu(261): error: no instance of overloaded function "atomicAdd" matches the argument list argument types are: (double *, double) detected during instantiation of "void VecQuant3MatMulKernel(const scalar_t *, const int *, scalar_t *, const scalar_t *, const scalar_t *, int, int, int, int) [with scalar_t=double]" (171): here

quant_cuda_kernel.cu(337): error: no instance of overloaded function "atomicAdd" matches the argument list argument types are: (double *, double) detected during instantiation of "void VecQuant4MatMulKernel(const scalar_t *, const int *, scalar_t *, const scalar_t *, const scalar_t *, int, int, int, int) [with scalar_t=double]" (283): here

quant_cuda_kernel.cu(409): error: no instance of overloaded function "atomicAdd" matches the argument list argument types are: (double *, double) detected during instantiation of "void VecQuant8MatMulKernel(const scalar_t *, const int *, scalar_t *, const scalar_t *, const scalar_t *, int, int, int, int) [with scalar_t=double]" (359): here

SO I changed all the integers to doubles... and it compiled with a bunch of warnings... Is this defeating the point or is this a valid solution? Nothing runs atm so I guess not.

[xiscoding]

I can not find torch/all.h or torch/python.h my fault

[qwopqwop200]

Unable to reproduce this error. Have you installed pytorch by any chance?

[xiscoding]

I just need to find the torch/all.h and torch/python.h files and put the full path in the include (#include <torch/all.h> ***NOT FOUND #include <torch/python.h> ***NOT FOUND #include <cuda.h> ***FOUND somehow I cant find the file in my system #include <cuda_runtime.h> ***FOUND I cant find the file in my system). Do you know where those files are? They are not in the same location where torch is installed.

(torch/csrc/api/include/torch/all.h) this is the path on the pytorch website (https://pytorch.org/cppdocs/api/file_torch_csrc_api_include_torch_all.h.html#definition-torch-csrc-api-include-torch-all-h)

(~/.local/lib/python3.10/site-packages/torch) this is where I looked. I dont have torch anywhere else except other environments

[antegral]

Unable to reproduce this error. Have you installed pytorch by any chance?

This error reproduces in the build process for Docker image. https://github.com/antegral/text-generation-webui-docker/blob/214fbabeee8779043dc5c15c84a5ed6dbdb1049a/Dockerfile https://github.com/antegral/text-generation-webui-docker/actions/runs/4386040919/jobs/7679549971

[antegral]

$ find / -name all.h
/opt/conda/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/all.h
/opt/conda/lib/python3.9/site-packages/torch/include/ATen/ops/all.h
/opt/conda/pkgs/pytorch-1.13.1-py3.9_cuda11.7_cudnn8.5.0_0/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/all.h
/opt/conda/pkgs/pytorch-1.13.1-py3.9_cuda11.7_cudnn8.5.0_0/lib/python3.9/site-packages/torch/include/ATen/ops/all.h

$ find / -name python.h
/opt/conda/lib/python3.9/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/common/python.h
/opt/conda/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/python.h
/opt/conda/pkgs/pytorch-1.13.1-py3.9_cuda11.7_cudnn8.5.0_0/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/python.h

$ find / -name cuda.h
/opt/conda/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/cuda.h
/opt/conda/include/cuda.h
/opt/conda/pkgs/pytorch-1.13.1-py3.9_cuda11.7_cudnn8.5.0_0/lib/python3.9/site-packages/torch/include/torch/csrc/api/include/torch/cuda.h
/opt/conda/pkgs/cuda-cudart-dev-11.7.99-0/include/cuda.h

$ find / -name cuda_runtime.h
/opt/conda/include/cuda_runtime.h
/opt/conda/pkgs/cuda-cudart-dev-11.7.99-0/include/cuda_runtime.h

[xiscoding]

Holy cow I am a fool I forgot to set visible devices... I have a 960 that does not support 8bit quantization...

CUDA_DEVICE_ORDER="PCI_BUS_ID" CUDA_VISIBLE_DEVICES="0,2" python setup_cuda.py install

set the visible devices to the devices that are supported by 8bit

Still a ton of errors but this is the successful run

running install /home/xdoestech/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( /home/xdoestech/.local/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn( running bdist_egg running egg_info writing quant_cuda.egg-info/PKG-INFO writing dependency_links to quant_cuda.egg-info/dependency_links.txt writing top-level names to quant_cuda.egg-info/top_level.txt /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.')) reading manifest file 'quant_cuda.egg-info/SOURCES.txt' writing manifest file 'quant_cuda.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_ext building 'quant_cuda' extension gcc -pthread -B /home/xdoestech/anaconda3/envs/textgen/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/xdoestech/anaconda3/envs/textgen/include -fPIC -O2 -isystem /home/xdoestech/anaconda3/envs/textgen/include -fPIC -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/TH -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/THC -I/home/xdoestech/anaconda3/envs/textgen/include -I/home/xdoestech/anaconda3/envs/textgen/include/python3.10 -c quant_cuda.cpp -o build/temp.linux-x86_64-cpython-310/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14 /home/xdoestech/anaconda3/envs/textgen/bin/nvcc -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/TH -I/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/THC -I/home/xdoestech/anaconda3/envs/textgen/include -I/home/xdoestech/anaconda3/envs/textgen/include/python3.10 -c quant_cuda_kernel.cu -o build/temp.linux-x86_64-cpython-310/quant_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14 quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:87:40: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:216:1: note: declared here 216 | DeprecatedTypeProperties & type() const { | ^ ~~ quant_cuda_kernel.cu:87:154: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^
/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:132:1: note: declared here 132 | inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) { | ^~~~~~~~~~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:87:938: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:959: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:983: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:1010: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:1036: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:87:1835: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:1856: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:1879: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:1905: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:87:1930: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 87 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:171:40: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:216:1: note: declared here 216 | DeprecatedTypeProperties & type() const { | ^ ~~ quant_cuda_kernel.cu:171:154: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^
/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:132:1: note: declared here 132 | inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) { | ^~~~~~~~~~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:171:939: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:960: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:984: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:1011: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:1037: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:171:1837: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:1858: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:1881: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:1907: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:171:1932: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 171 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:283:40: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:216:1: note: declared here 216 | DeprecatedTypeProperties & type() const { | ^ ~~ quant_cuda_kernel.cu:283:154: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^
/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:132:1: note: declared here 132 | inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) { | ^~~~~~~~~~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:283:939: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:960: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:984: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:1011: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:1037: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:283:1837: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:1858: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:1881: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:1907: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:283:1932: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 283 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:359:40: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:216:1: note: declared here 216 | DeprecatedTypeProperties & type() const { | ^ ~~ quant_cuda_kernel.cu:359:154: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^
/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:132:1: note: declared here 132 | inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) { | ^~~~~~~~~~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:359:939: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:960: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:984: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:1011: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:1037: warning: ‘T* at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:359:1837: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:1858: warning: ‘T* at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:1881: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:1907: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ quant_cuda_kernel.cu:359:1932: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data<T>() is deprecated. Please use Tensor.data_ptr<T>() instead. [-Wdeprecated-declarations] 359 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:238:1: note: declared here 238 | T * data() const { | ^ ~~ creating build/lib.linux-x86_64-cpython-310 g++ -pthread -B /home/xdoestech/anaconda3/envs/textgen/compiler_compat -shared -Wl,-rpath,/home/xdoestech/anaconda3/envs/textgen/lib -Wl,-rpath-link,/home/xdoestech/anaconda3/envs/textgen/lib -L/home/xdoestech/anaconda3/envs/textgen/lib -Wl,-rpath,/home/xdoestech/anaconda3/envs/textgen/lib -Wl,-rpath-link,/home/xdoestech/anaconda3/envs/textgen/lib -L/home/xdoestech/anaconda3/envs/textgen/lib build/temp.linux-x86_64-cpython-310/quant_cuda.o build/temp.linux-x86_64-cpython-310/quant_cuda_kernel.o -L/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/torch/lib -L/home/xdoestech/anaconda3/envs/textgen/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda_cu -ltorch_cuda_cpp -o build/lib.linux-x86_64-cpython-310/quant_cuda.cpython-310-x86_64-linux-gnu.so creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-cpython-310/quant_cuda.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg creating stub loader for quant_cuda.cpython-310-x86_64-linux-gnu.so byte-compiling build/bdist.linux-x86_64/egg/quant_cuda.py to quant_cuda.cpython-310.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying quant_cuda.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying quant_cuda.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying quant_cuda.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying quant_cuda.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt zip_safe flag not set; analyzing archive contents... pycache.quant_cuda.cpython-310: module references file creating dist creating 'dist/quant_cuda-0.0.0-py3.10-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing quant_cuda-0.0.0-py3.10-linux-x86_64.egg removing '/home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/quant_cuda-0.0.0-py3.10-linux-x86_64.egg' (and everything under it) creating /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/quant_cuda-0.0.0-py3.10-linux-x86_64.egg Extracting quant_cuda-0.0.0-py3.10-linux-x86_64.egg to /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages quant-cuda 0.0.0 is already the active version in easy-install.pth

Installed /home/xdoestech/anaconda3/envs/textgen/lib/python3.10/site-packages/quant_cuda-0.0.0-py3.10-linux-x86_64.egg Processing dependencies for quant-cuda==0.0.0 Finished processing dependencies for quant-cuda==0.0.0