sdc icon indicating copy to clipboard operation
sdc copied to clipboard

Does one need to (can?) redefine functions from other standard libraries?

Open laszlosandor opened this issue 5 years ago • 1 comments

As far as I understand, much of pandas is explicitly supported. Does it mean that other Intel goodies are not supported out of the box? I mean these.

Does it matter if I manually add @hpat.jit to those functions? (Though if it were that easy, you probably would have done already…)

FWIW, the econ crowd would love it if binscatter got magically faster. (Though the Python implementation is lagging behind the Stata/R reboot by now.)

laszlosandor avatar Jun 13 '19 09:06 laszlosandor

HPAT supports a subset of numpy and pandas (https://intellabs.github.io/hpat-doc/dev/source/supportedpandas.html). Because it is based on numba it's use of MKL is similar to numba.

HPAT/numba will compile any python function to native as long as it uses features supported by numba/hpat.

Notice that HPAT auto-distributes data-frames. Getting this feature right for arbitrary functions is of course additional work since you have to tell HPAT how to operate in a distributed setup.

We'd be happy to help if you'd like to try.

fschlimb avatar Jun 27 '19 12:06 fschlimb