An alternative to tensor adaptor

[IvanaGyro]

Per #500 and several internal meetings, we decided to design a tensor adaptor to bridge the tensor implementation from other libraries. The purpose is to include the feature provided in other libraries, such as PyTorch's automatic differentiation (AD).

However, making an adaptor to connect other tensor implementations is not the end of the story because we also have to update all code, including linear algebra, that uses tensor. Reimplementing the components interesting with tensor is a heavy job. #306 is just a start.

The first alternative to designing a tensor adaptor is replacing everything equal and lower than the tensor level with the library, which provides the feature we want and also fits our requirements. PyTorch seems to be the most frequently mentioned candidate. There is another alternative for referring if the first alternative is not an option at this moment.

@hunghaoti raised the second alternative. We can implement a protocol to convert Cytnx's tensor to another tensor implementation so that the users of Cytnx can use the feature provided by the other libraries. After manipulating the data with the feature in the other libraries, the users can convert tensor back to Cyntx's implementation. DLPack is the protocol widely used by Python libraries like PyTorch, Numpy, CuPy, Tensorflow, etc., and it is also a portocal required by the Python array API. As a result, supporting DLPack portocal can enable our users to transfer Cytnx's tensor to/from other libraries.

However, the limitation of the second alternative is that the users cannot apply the features, like AD, provided by other libraries at a level higher than tensor like UniTensor because the calculation of UniTensor still relies on our linear algebra implementation.

Do you think which alternative is better?

[kaihsin]

I don't see how we can avoid doing that. If we want to support different backend. To add a backend one eventually will need to specify which target API to call right?

// interface
template<backend>
class Tensor{
     virtual Tensor<backend> matmul(const Tensor<backend> tn);
     virtual Tensor<backend> svd(const Tensor<backend> tn); 
};

// numpy backend 
Tensor<numpy>::matmul(const Tensor<numpy> tn){
      return numpy::matmul(*this, tn); // numpy api 
}

//pytorch backend
Tensor<pytorch>::matmul(const Tensor<pytorch> tn){
    return torch:mm(*this, tn);  // pytorch api
}

// other backend 

I would say the first solution will probably be better. We should also provide second solution for things that we does not wrapped.