[ITensors] [ENHANCEMENT] In-place operations with unequal dimensions of indices.

[ArtemStrashko]

Is your feature request related to a problem? Please describe.

Allocations reduction is a key to writing performant code. While, e.g., an operation like A .= B .* C is available, it assumes that A and B*C have the same indices, which, among other things, means that dimensions of indices must be the same. To write optimal code, sometimes it would be useful to lift this requirement.

For example, suppose we do a 2-site DMRG update. This requires merging m_combined = m_{i} * m_{i+1}, updating it m_combined -> m_combined^*, splitting it m_combined^* = U * S * V, cutting S and then updating m_{i} -> U and m_{i+1} = S * V, i.e. creating new m_{i} and m_{i+1} tensors. For some applications, to reduce allocations, it would be useful to be able to pre-allocate large enough m_i and update them in-place. While it is simple in pure Julia, I wonder how hard it would be to have it in ITensors.

Describe the solution you'd like

It would be useful to have some helper functions allowing for in-place operations with tensors with indices of unequal dimensions.

Describe alternatives you've considered

So far, alternatives have been writing code in pure Julia and miss rich ITensors functionality or to enjoy simplicity and power of ITensors at a cost of extra allocations.

[mtfishman]

We have been discussing a general buffer system which can help automate pre-allocating data to decrease allocations in tensor operations (@kmp5VT was testing out a prototype in #1011), while being more flexible than the current interface. Definitely a priority on our ever-growing todo list.