how is this effort related to intel python distribution (idp)

[petacube]

i am curious about relationship between hpat project and https://software.intel.com/en-us/distribution-for-python. can hpat be enhanced by work intel is doing with numba and icc compiler?

both projects are based on llvm and both enhance numba. hpat looks like port of previous effort on julia parallel accelerator.

[ehsantn]

HPAT is based on Numba (our ParallelAccelerator in Numba) but is focused on distributed-memory analytics. Yes, we ported HPAT and ParallelAccelerator from Julia.

[petacube]

i see. so looks like to me intel work on numba with intel compiler is focused on multithreaded performance with intel own libs - mkl, gdal etc, whereas your is more on focused on distributed aspects somewhat overlapping with dask/distributed project as well.

[ehsantn]

HPAT is focused on distributed analytics but leverages Numba to large extent. Intel compiler is not used in Numba (LLVM is the backend). We are working on various aspects of Numba though.

[saulshanabrook]

Would this project ever be included in Numba?

[ehsantn]

We have discussed it a couple of times. Doesn't sound like there is a lot of benefit in doing that.

[abhi-84]

By adding SDC with daal4py, would it gives improved execution time?Someone have tested?