Full support of Intel VML functions

[FrancescAlted]

From [email protected] on February 21, 2012 14:11:39

Numexpr is currently supporting a number of functions from the Intel Math libraries.

For calculations involving statistics, peak analysis and other areas a full support of all functions from the VML mathematical functions would greatly enhance the versatility of numexpr. Especially the random number generators, error function, gamma function and rounding functions are still not supported.

Original issue: http://code.google.com/p/numexpr/issues/detail?id=74

[mtrbean]

I am working on something that requires gammaln and betaln and I'm interested in adding support for these functions. I'm interested in contributing but will need some directions for me to get started

[FrancescAlted]

I think the best is to see how others added support for other functions. Here you have a couple of examples:

https://github.com/pydata/numexpr/pull/125 https://github.com/pydata/numexpr/pull/135

I think the former is closer to what you want.

[mtrbean]

These functions are under scipy.special. Is it ok to have scipy as a dependency?

[gdementen]

Probably not (I would vote against it, but I am not the maintainer). Besides, is there a C API for these functions? You cannot call a Python function from numexpr (in a numexpr expression).

On Mon, Jun 13, 2016 at 11:48 PM, Eric Wong [email protected] wrote:

These functions are under scipy.special. Is it ok to have scipy as a dependency?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/numexpr/issues/74#issuecomment-225719036, or mute the thread https://github.com/notifications/unsubscribe/AAyyXKUqBP5v72BttezhijkECupsqKQRks5qLdAWgaJpZM4Bbm1v .

[FrancescAlted]

Yes, I agree with all @gdementen has expressed.

[robbmcleod]

Scipy has a pretty permissible license, so if the functions are implemented in C one could more-or-less cut and paste them?

On Tue, Jun 14, 2016 at 11:22 AM, FrancescAlted [email protected] wrote:

Yes, I agree with all @gdementen https://github.com/gdementen has expressed.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pydata/numexpr/issues/74#issuecomment-225826437, or mute the thread https://github.com/notifications/unsubscribe/AFuMmcyHfLfObKRcXLJYdHlKRL2x6la9ks5qLnLPgaJpZM4Bbm1v .

Robert McLeod, Ph.D. Center for Cellular Imaging and Nano Analytics (C-CINA) Biozentrum der Universität Basel Mattenstrasse 26, 4058 Basel Work: +41.061.387.3225 [email protected] [email protected] [email protected] [email protected]

[FrancescAlted]

Yes, I don't see problems if the functions are in C and can be embedded in numexpr. However, one thing that worries me is if there is much point in adding too much specific functions, and gammaln and betaln looks like they fall in this category.

[RoyiAvital]

By the way, if I install Anaconda's Numpy which comes with Intel MKL (Or any Numpy compiled with MKL), can numexpr use that for VML?

[gdementen]

@RoyiAvital This depends how your numexpr has been compiled. AFAIK, most recent versions of numexpr on Anaconda are linked against the MKL.

[RoyiAvital]

Any way getting this working on PIP as well without the need to recompile anything?

I could see it on Python Extension Packages for Windows - Christoph Gohlke which compiles packages like that.

[gdementen]

@RoyiAvital I suppose the binaries on pypi were not linked against the MKL, so you will have to compile yourself. But since you are using anaconda, why don't you install numexpr from there too?

[RoyiAvital]

I can mange it by myself (Or any Anaconda user for that matter). Just thought it would be easier to more users if it was offered on PIP linked to MKL.

I wish someone would make available on PIP all those MKL accelerated packages.

[RoyiAvital]

Is there a test to verify numexpr uses MKL (VML) on a given configuration?

[FrancescAlted]

Yes:

$ python -c"import numexpr; numexpr.print_versions()"
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version:   2.6.4
NumPy version:     1.14.2
Python version:    3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
Platform:          darwin-x86_64
AMD/Intel CPU?     False
VML available?     True
VML/MKL version:   Intel(R) Math Kernel Library Version 2018.0.2 Product Build 20180127 for Intel(R) 64 architecture applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

[rootsmusic]

Can the latest version still enable Intel VML? VML's README.md says, "This project will no longer be maintained by Intel."

When I run numexpr.test(), it outputs: "VML available? False". However, I'm unsure if it's indicating my platform is missing that library or if numexpr hasn't enabled VML (and that library isn't missing). Thanks.

[github-actions[bot]]

Message to comment on stale issues. If none provided, will not mark issues stale