pysheds icon indicating copy to clipboard operation
pysheds copied to clipboard

Re-evaluate Numba performance

Open groutr opened this issue 2 years ago • 2 comments
trafficstars

I think the usage of numba for certain functions needs to be re-evaluated. Sometimes the numba version is much slower than just doing things in numpy.

Example:

def np_vec(affine, x, y):
    a, b, c, d, e, f, _, _, _ = affine
    new_x = x * a + y * b + c
    new_y = x * d + y * e + f
    return new_x, new_y
x = np.random.rand(100)
y = np.random.rand(100)

%timeit _affine_map_vec_numba((1, 2, 3, 4, 5, 6, 7, 8, 9), x, y)
82.8 µs ± 13 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

%timeit np_vec((1, 2, 3, 4, 5, 6, 7, 8, 9), x, y)
17.4 µs ± 1.04 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

groutr avatar Oct 17 '23 15:10 groutr

Is this related to #230?

mdbartos avatar Oct 30 '23 19:10 mdbartos

@mdbartos It is tangentially related. #230 addresses the issue of numba being less efficient due to the strict typing. This ticket is to address the set of cases where numpy is just simply faster than using numba.

The case presented above is the code for a scalar function. The array case uses numba, but for such a simple computation, using numpy produces a faster result. Numexpr is a library that could be of use here to parallelize the computation in a memory efficient way.

groutr avatar Oct 30 '23 20:10 groutr