Deal with potential clobber between PyTorch's copy of ATen and a standalone ATen

[ezyang]

Both PyTorch and ATen (standalone) produce libATen.so files. This is hazardous because if they are not ABI compatible (and they probably are not), you will get extremely hard to diagnose errors if one clobbers another.

Unfortunately, on the PyTorch side, libATen.so cannot be statically linked into Torch's main _C.so, because cffi plugins for PyTorch may wish to interact with ATen directly. (Actually, how easy is it to actually do this? As ATen is a C++ library, any FFI code must be very careful to get C++ ABI compatibility...)

One possibility is to give PyTorch and standalone ATen distinct symbol names and library names.

Another possibility is to "deprecate" standalone ATen installation: to get ATen, you must install PyTorch (solving the duplicate ATen problem.) TH headers get installed to $CONDA_PREFIX/lib/python3.6/site-packages/torch/lib/include/ which can be used by users (albeit with some difficulty.) One hazard is that you must still make sure your C++ compiler is ABI compatible with the ATen build.

[gchanan]

is this fixed now?

Also...should we move these issues to PyTorch github?

[zdevito]

It is partially fixed. Yet we should move the issues we still want to fix over to pytorch.