Dynamic xt::concatenate

[arakhmati]

Currently, xt::concatenate takes xt::tuple as its first argument which means that the number of input arrays to xt::concatenate has to be specified during the compile time.

Would it be possible to make a dynamic version of xt::concatenate by overloading it to takestd::vector as its first argument? I

[JohanMabille]

Hi this is possible but requires some refactoring of the current implementation. Is this something you would like to tackle?

[arakhmati]

I would be interested in tackling this. Can you please briefly explain what refactoring is needed?

[JohanMabille]

To implement a dynamic concatenate, you have to implement a dynamic equivalent of https://github.com/QuantStack/xtensor/blob/f66ffca22d553e593319754d98f858725d14f243/include/xtensor/xbuilder.hpp#L429 Now the problem with dynamic containers is that they cannot store objects of different types, so either we decide to have a dynamic concatenate that can operate on objects of the same type (i.e. it is not possible to concatenate xarray and xview for instance), or you have to go with type erasure. We already have an expression holder but it is quite incomplete, you'll probably have to add access operators.

The first solution is easier to implement and the code will be faster since you don't have to go through virtual call for element access, but this is at the cost of genericity.

Whatever the solution you choose to implement, the dynamic version of concatenate should be able to work with other containers from xtensor (i.e. uvector, svector, etc, not only std::vector).

[mirmohammad]

Since concat does not take list or vector as input, is the best workaround to iteratively concatenating two elements from the list?

[JohanMabille]

That would be highly inefficient, but that can be a workaround.

[mirmohammad]

Well, I realized that you can actually get a similar logic by using xt::from_indices and then transposing it. Specifically, the problem I was facing using xt::nonzero can be solved by xt::transpose(xt::from_indices(xt::nonzero(some_array))). This will be equivalent of np.stack(np.nonzero(some_array), axis=1)

[jf99]

I tried to implement an overload of xt::concatenate that takes a vector, but apparently I'm not experienced enough in meta-programming in order to see through xtensor's internals. So I implemented a simple non-meta-programming version that I am happy to share with others: https://gist.github.com/jf99/5b79b76a8ed721cc6b1a6d24ccb4f6ad

It's not optimized but it's far more efficient than iterating over a vector and adding each vector element using the tuple-concatenate.