Implementing Numba in Python and Apple Accelerate (matrix processor support) in Numpy?

[stenerikbjorling]

What's your use case?

By using Numba and the Apple Accelerate (matrix) compiler directives for Numpy the performance might increase for larger data sets and longer operations.

Just some thoughts...

Want to use Orange3 to be a good analysis tool for very large datasets and complex calculations.

What's your proposed solution?

Are there any alternative solutions?

[markotoplak]

Thank you for the initiative.

Numba was a dependency of Orange because one of Orange's dependencies depended on it. That was a few years ago. We do like numba from a programming perspective, but it made distribution harder: occasionally, there were dependency bugs and it also resulted in ~100MB larger downloads. Therefore we decided to remove it as a dependency. The computationally intensive parts are of Orange work through numpy anyway. Orange also uses some custom helpers in cython.

As far as compilers go: Orange's code that needs to be compiled is quite tiny. More important is how dependencies, primarily numpy, are compiled. Here, we use official packages from either PyPi or conda-forge when packaging Orange for end users. I believe these are now optimized quite well. Even if they are slightly suboptimal, we'd like to avoid building custom packages.

I remember that ~5 years ago the official NumPy packages were quite slow (thus, Orange was much slower on Macs than on Linux or Windows), but then became optimized. Lately, I do not see big differences, but was not measuring them either for quite some time. Do you perhaps make any measurements of how faster Orange could be by swapping a compiler (and libraries) used in building NumPy?