Questions on migration from numpy to xtensor.

[OleksiiMatiash]

I'm trying to rewrite simple console python\numpy app to Qt app with UI, and xtensor seems to be the right choice to replace numpy. However, I'm unable to understand how to replace some parts of python-numpy with it. Here is full code I'm trying to rewrite: https://pastebin.com/BUYZwCsn

And here is first section that stopped me:

image: : ndarray[uint16]
...
r1 = image[::2].reshape((-1))
r2 = image[1::2].reshape((-1))
channels = np.empty((4, image.size // 4), dtype = uint16)
channels[0] = r1[0::2]
channels[1] = r1[1::2]
channels[2] = r2[0::2]
channels[3] = r2[1::2]

It transformed to

...
auto activeAreaImageR1 = xt::view(activeAreaImage, xt::all(), 2);
xt::reshape_view(activeAreaImageR1, { -1 });

xt::xarray<uint16_t> activeAreaImageR2 = xt::view(activeAreaImage, xt::range(1, 2));
xt::reshape_view(activeAreaImageR2, { -1 });

auto channels = xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 });

... and what's next? How to implement assigning channels[0] = ...?

Another question is: how can I pass\return xtensor arrays to\from functions? I.e. how to implement something like this without overhead?

def getActiveAreaPixels(image: ndarray[uint16], height: int, width: int, activeArea: [int]) -> ndarray[uint16]:
    return image.reshape(height, width)[activeArea[0]:activeArea[2], activeArea[1]:activeArea[3]]

Thank you in advance.

BTW, maybe there is discord or any other chat for such questions? I believe creating issues is not the best way..

[argman]

you can assign views to views, like

xt::view(channels, 0) = xt::view(....)

[hassanromhuda]

you can assign views to views, like

xt::view(channels, 0) = xt::view(....)

I can't get something like this to work. See https://github.com/xtensor-stack/xtensor/issues/2748. Any ideas?

[JohanMabille]

channels needs to be an evaluated expression for having this work.

[OleksiiMatiash]

@argman

you can assign views to views, like

xt::view(channels, 0) = xt::view(....)

If I get this correctly, it compiles, but crashes on assignment:

auto channels = xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 });
auto ch0 = xt::view(activeAreaImageR1, xt::range(0, 2));
auto ch01d = xt::reshape_view(ch0, { activeAreaImageHeight * activeAreaImageWidth / 4 });
xt::view(channels, 0) = ch0;

@JohanMabille

channels needs to be an evaluated expression for having this work.

Sorry, I probably didn't get the idea, because this does not work:

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
channels[0] = xt::eval(xt::view(activeAreaImageR1, xt::range(0, 2)));

With any combinations of xt::eval it tells me this:

[JohanMabille]

The first snippet should not build since you still try to assign to a view on an unevaluated expression.

Regading the second one, be carefull that operator[] in xtensor is different from operator[] in numpy, see https://github.com/compiler-research/xeus-cpp/actions/runs/6892192995/job/18748906995?pr=19 for the different ways to access elements in xensor expressions. The following should work:

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
xt::view(channels, 0) = xt::eval(xt::view(activeAreaImageR1, xt::range(0, 2)));

(assuming the shapes are compatible).

[OleksiiMatiash]

The first snippet should not build since you still try to assign to a view on an unevaluated expression.

First variant compiles, but crashes on execution of last line.

see https://github.com/compiler-research/xeus-cpp/actions/runs/6892192995/job/18748906995?pr=19

Wrong link, probably?

This compiles (added reshape), but crashes on execution

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
xt::view(channels, 0) = xt::eval(xt::reshape_view(xt::view(activeAreaImageR1, xt::range(0, 2)), { activeAreaImageHeight * activeAreaImageWidth / 4 }));

[JohanMabille]

Wrong link, probably?

Indeed, sorry for that, here is the correct one: https://xtensor.readthedocs.io/en/latest/expression.html#element-access

This compiles (added reshape), but crashes on execution

Can you print the shapes of both expressions in the last statement? No need for xt::eval on the right hand side.

[OleksiiMatiash]

Can you print the shapes of both expressions in the last statement? No need for xt::eval on the right hand side.

If I got you correctly (sorry if no), this

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
auto reshapedView = xt::reshape_view(xt::view(activeAreaImageR1, xt::range(0, 2)), { activeAreaImageHeight * activeAreaImageWidth / 4 });
auto shape1 = reshapedView.shape();
auto shape2 = xt::view(channels, 0).shape();

gives this:

[JohanMabille]

OK, so the shpaes are compatible, I would try the following:

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
auto image_view = xt::eval(xt::view(activeAreaImageR1, xt::range(0, 2)));
auto reshapedView = xt::reshape_view(image_view, { activeAreaImageHeight * activeAreaImageWidth / 4 });
xt::view(channels, 0)  = reshaped_view;

and the following:

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
auto image_view = xt::eval(xt::view(activeAreaImageR1, xt::range(0, 2)));
auto reshapedView = xt::eval(xt::reshape_view(image_view, { activeAreaImageHeight * activeAreaImageWidth / 4 }));
xt::view(channels, 0)  = reshaped_view;

to see where it crashes. You should also check how activeAreaImageR1 is initialized, and its shape.

[OleksiiMatiash]

This

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
auto image_view = xt::eval(xt::view(activeAreaImageR1, xt::range(0, 2)));
auto reshapedView = xt::reshape_view(image_view, { activeAreaImageHeight * activeAreaImageWidth / 4 });
xt::view(channels, 0)  = reshaped_view;

crashes on the last line:

And this:

auto channels = xt::eval(xt::empty<uint16_t>({ 4, activeAreaImageHeight * activeAreaImageWidth / 4 }));
auto image_view = xt::eval(xt::view(activeAreaImageR1, xt::range(0, 2)));
auto reshapedView = xt::eval(xt::reshape_view(image_view, { activeAreaImageHeight * activeAreaImageWidth / 4 }));
xt::view(channels, 0)  = reshaped_view;

on this line:

auto reshapedView = xt::eval(xt::reshape_view(image_view, { activeAreaImageHeight * activeAreaImageWidth / 4 }));

You should also check how activeAreaImageR1 is initialized, and its shape.

Full code:

uint16_t* sourceBuffer = new uint16_t[sourceFileInfo->metadata->dataSize / sizeof(uint16_t)];

FILE* fileptr;
fileptr = fopen(filePath.toStdString().c_str(), "rb");
if (fileptr == nullptr) return;

fseek(fileptr, sourceFileInfo->metadata->dataOffset, SEEK_SET);
fread(sourceBuffer, sizeof(uint16_t), sourceFileInfo->metadata->dataSize / sizeof(uint16_t), fileptr);
fclose(fileptr);

std::vector shape = { sourceFileInfo->metadata->dataSize / sizeof(uint16_t) };
QList<int> activeArea = sourceFileInfo->metadata->activeArea;
int activeAreaImageHeight = activeArea[2] - activeArea[0];
int activeAreaImageWidth = activeArea[3] - activeArea[1];

auto source = xt::adapt(sourceBuffer, sourceFileInfo->metadata->dataSize / sizeof(uint16_t), xt::no_ownership(), shape);

source.reshape({ sourceFileInfo->metadata->imageHeight, sourceFileInfo->metadata->imageWidth });
auto activeAreaImage = xt::view(source, xt::range(activeArea[0], activeArea[2]), xt::range(activeArea[1], activeArea[3]));

auto activeAreaImageR1 = xt::view(activeAreaImage, xt::all(), 2);
xt::reshape_view(activeAreaImageR1, { -1 });

xt::xarray<uint16_t> activeAreaImageR2 = xt::view(activeAreaImage, xt::range(1, 2));
xt::reshape_view(activeAreaImageR2, { -1 });

and then the code with problems from original question