Anaconda fail to build with "--cpp_ext" and "--cuda_ext" options

[BurguerJohn]

It work fine without those options.

Error:

E:/Miniconda3/lib/site-packages/torch/include\c10/util/BFloat16.h(51): warning: calling a __host__ function from a __host__ __device__ function is not allowed
          detected during:
            instantiation of class "std::is_const<_Ty> [with _Ty=std::vector<at::Tensor, std::allocator<at::Tensor>>]"
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\type_traits(683): here
            instantiation of "const __nv_bool std::is_const_v [with _Ty=std::vector<at::Tensor, std::allocator<at::Tensor>>]"
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\xmemory0(943): here
            instantiation of class "std::allocator<_Ty> [with _Ty=std::vector<at::Tensor, std::allocator<at::Tensor>>]"
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\xmemory0(903): here
            instantiation of class "std::allocator_traits<_Alloc> [with _Alloc=std::allocator<std::vector<at::Tensor, std::allocator<at::Tensor>>>]"
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\xmemory0(916): here
            instantiation of type "std::_Rebind_alloc_t<std::allocator<std::vector<at::Tensor, std::allocator<at::Tensor>>>, std::vector<at::Tensor, std::allocator<at::Tensor>>>"
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\vector(360): here
            instantiation of class "std::_Vec_base_types<_Ty, _Alloc> [with _Ty=std::vector<at::Tensor, std::allocator<at::Tensor>>, _Alloc=std::allocator<std::vector<at::Tensor, std::allocator<at::Tensor>>>]"
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\vector(405): here
            instantiation of class "std::_Vector_alloc<_Alloc_types> [with _Alloc_types=std::_Vec_base_types<std::vector<at::Tensor, std::allocator<at::Tensor>>, std::allocator<std::vector<at::Tensor, std::allocator<at::Tensor>>>>]"
C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\vector(625): here
            instantiation of class "std::vector<_Ty, _Alloc> [with _Ty=std::vector<at::Tensor, std::allocator<at::Tensor>>, _Alloc=std::allocator<std::vector<at::Tensor, std::allocator<at::Tensor>>>]"
csrc/multi_tensor_scale_kernel.cu(118): here

C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/include\type_traits(1271): error: static assertion failed with "You've instantiated std::aligned_storage<Len, Align> with an extended alignment (in other words, Align > alignof(max_align_t)). Before VS 2017 15.8, the member type would non-conformingly have an alignment of only alignof(max_align_t). VS 2017 15.8 was fixed to handle this correctly, but the fix inherently changes layout and breaks binary compatibility (*only* for uses of aligned_storage with extended alignments). Please define either (1) _ENABLE_EXTENDED_ALIGNED_STORAGE to acknowledge that you understand this message and that you actually want a type with an extended alignment, or (2) _DISABLE_EXTENDED_ALIGNED_STORAGE to silence this message and get the old non-conformant behavior."
          detected during:
            instantiation of class "std::_Aligned<_Len, _Align, double, false> [with _Len=16ULL, _Align=16ULL]"
(1291): here
            instantiation of class "std::_Aligned<_Len, _Align, int, false> [with _Len=16ULL, _Align=16ULL]"
(1298): here
            instantiation of class "std::_Aligned<_Len, _Align, short, false> [with _Len=16ULL, _Align=16ULL]"
(1305): here
            instantiation of class "std::_Aligned<_Len, _Align, char, false> [with _Len=16ULL, _Align=16ULL]"
(1312): here
            instantiation of class "std::aligned_storage<_Len, _Align> [with _Len=16ULL, _Align=16ULL]"
csrc/multi_tensor_scale_kernel.cu(25): here
            instantiation of "void load_store(T *, T *, int, int) [with T=float]"
csrc/multi_tensor_scale_kernel.cu(64): here
            instantiation of "void ScaleFunctor<in_t, out_t>::operator()(int, volatile int *, TensorListMetadata<2> &, float) [with in_t=float, out_t=float]"
e:\downloads\dain-master\dain-master\dain\my dain\apex\csrc\multi_tensor_apply.cuh(37): here
            instantiation of "void multi_tensor_apply_kernel(int, volatile int *, T, U, ArgTypes...) [with T=TensorListMetadata<2>, U=ScaleFunctor<float, float>, ArgTypes=<float>]"
e:\downloads\dain-master\dain-master\dain\my dain\apex\csrc\multi_tensor_apply.cuh(106): here
            instantiation of "void multi_tensor_apply<depth,T,ArgTypes...>(int, int, const at::Tensor &, const std::vector<std::vector<at::Tensor, std::allocator<at::Tensor>>, std::allocator<std::vector<at::Tensor, std::allocator<at::Tensor>>>> &, T, ArgTypes...) [with depth=2, T=ScaleFunctor<float, float>, ArgTypes=<float>]"
csrc/multi_tensor_scale_kernel.cu(124): here

1 error detected in the compilation of "C:/Users/Gabriel/AppData/Local/Temp/tmpxft_000045e0_00000000-14_multi_tensor_scale_kernel.compute_70.cpp1.ii".

[sbuser]

I have the same issue using a pytorch compiled from source and MSVC 2017.

With -vcvars_ver=14.16 there are unresolvable alignment errors as shown above (the #define doesn't seem to work for me)

With -vcvars_ver=14.11 (which pytorch was compiled with) there are a slew of undefined identifiers.

(ninja was also compiled from source)

[zeyus]

For anyone else in future despair who arrives here in need of an updated solution (the other forked repo is fairly old now), I have created a gist with a diff to get it working with:

• CUDA 11.8
• Windows 11 Pro 10.0.23516
• Apex master commit 2958e06e094146798ae42a748a77a509fd4695a0: 30 July 2023
• Mamba (anaconda) env with
• Python 3.11
• pip 23.2.1
• PyTorch version: 2.1.0.dev20230801+cu118
• VS 2022

https://gist.github.com/zeyus/44aca39866d044a5ad1e8711e14ba9c1