pythran icon indicating copy to clipboard operation
pythran copied to clipboard

Indexing mismatch numpy/pythran when indexing 3D array with 2D bool array

Open cycomanic opened this issue 4 years ago • 2 comments

I just encountered an issue with "advanced" indexing. In fact there might be two issues.

The code below yields completely different results for numpy and pythran

#pythran export avgpixel(float64[:,:,3], float64)
def avgpixel(img, avg):
    out = np.zeros_like(img)
    o = np.mean(img, axis=-1)
    out[o>avg] =  255
    return out

I moved the below compile error to #1717 because I'm not sure it directly related.

import numpy as np

#pythran export avgpixel(float64[:,:,3], float64)
def avgpixel(img, avg):
    out = np.zeros_like(img)
    o = np.mean(img, axis=-1)
    idx = np.where(o>avg)
    out[idx] = 255
    return out

I get a compile error in pythran (see below)

In file included from /home/jschrod/PycharmProjects/pythran/pythran/pythonic/include/types/ndarray.hpp:39:
/home/jschrod/PycharmProjects/pythran/pythran/pythonic/include/types/numpy_gexpr.hpp:533:5: error: static_assert failed due to requirement 'utils::all_of<true, false>::value' "all slices are valid"
    static_assert(
    ^
/tmp/tmp1695bo21.cpp:70:5: note: in instantiation of template class '(anonymous namespace)::pythonic::types::numpy_gexpr<(anonymous namespace)::pythonic::types::ndarray<double, (anonymous namespace)::pythonic::types::array_base<long, 3, (anonymous namespace)::pythonic::types::tuple_version>>, (anonymous namespace)::pythonic::types::contiguous_normalized_slice, (anonymous namespace)::pythonic::types::ndarray<long, (anonymous namespace)::pythonic::types::array_base<long, 1, (anonymous namespace)::pythonic::types::tuple_version>>>' requested here
    out[idx] = 255L;
    ^
/tmp/tmp1695bo21.cpp:81:78: note: in instantiation of function template specialization '__pythran_test_average::avgpixel::operator()<(anonymous namespace)::pythonic::types::ndarray<double, (anonymous namespace)::pythonic::types::pshape<long, long, std::integral_constant<long, 3>>> &, double &>' requested here
                                auto res = __pythran_test_average::avgpixel()(img, avg);
                                                                             ^
1 error generated.
WARNING: Compilation error, trying hard to find its origin...
WARNING: Nop, I'm going to flood you with C++ errors!
CRITICAL: Cover me Jack. Jack? Jaaaaack!!!!

This is on latest master.

cycomanic avatar Jan 29 '21 21:01 cycomanic

To clarify I called the avgpixel in the case where it differs with:

x = np.random.random(size=(1024,1024,3))
out = avgpixel(x, 0.5)

cycomanic avatar Jan 29 '21 21:01 cycomanic

I can further simplify the problem to the following function:

#pythran export index1(float64[:,:,3], bool[:,:])
def index1(x, idx):
    return x[idx]

called with

ins] In [16]: x=np.random.randn(10*10*3).reshape(10,10,3)
[ins] In [17]: o=np.mean(x, axis=-1)
[ins] In [18]: ii = o>0.5
[ins] In [20]: x[ii].shape
Out[20]: (15, 3)
[ins] In [25]: index1(x, ii).shape
Out[25]: (15,)

As you see there is a shape mismatch, however the pythran array is not one of the 3 subarray of the numpy one.

cycomanic avatar Jan 31 '21 20:01 cycomanic