Support for tensors with 0-length dimensions

[michaeleisel]

trafficstars

Operations on tensors with zero-length dimensions are supported in other libraries such as PyTorch, and would be nice to have support for here. For example, when I multiply a 0-by-K tensor with a K-by-N tensor, I want a 0-by-N tensor back. Instead, I currently get a mysterious CUDA error when I try. In fact, generally for operations with a 0-length dimension, we don't need to use any CUDA/BLAS/other computational libraries at all, it's just a matter of constructing a tensor with the appropriate shape. Here are some examples of situations where it'd be nice to have support, and how it could be done:

• Multiplication of two tensors, where just the inner dimension is 0: create a tensor of all 0's, with the appropriate shape from the outer dimensions
• Multiplication of two tensors, where an outer dimension is 0: return a tensor with the appropriate shape from the appropriate outer dimensions, with no values needing to be generated, since it has zero elements
• Addition of two tensors: simply return one of the tensors, as it is equal to the correct answer