numdifftools icon indicating copy to clipboard operation
numdifftools copied to clipboard

Published 20 hours ago • verified publisher icon pbrod

Higher-order multivariate derivative

Open soerenwolfers opened this issue 6 years ago • 3 comments

Assume f is a function that consumes numpy arrays of shape N x d and returns arrays of the form N (i.e., it is a map from R^d to R and allows for vectorized calls).

What is the most elegant way to compute $\partial^{n_0}{0}\dots\partial^{n_d}{d}f$ using numdifftools, if possible in a vectorized fashion as well?

soerenwolfers avatar Jul 08 '19 10:07 soerenwolfers

Currently there is no easy way to calculate the general multivariate partial derivative like this \begin{align*} \frac{\partial^{n_0}}{\partial x_0^{n_0}}\dots\frac{\partial^{n_d} }{\partial x_d^{n_d}} f(x_0, \ldots, x_d) \end{align*}

with numdifftools. However, for n=1 and n=2 you can easily calculate partial derivatives \begin{align*} \frac{\partial^{n} }{\partial x_0^{n}} f(x_0, x_1,\ldots, x_d), \frac{\partial^{n} }{\partial x_1^{n}} f(x_0, x_1,\ldots, x_d), \ldots, \frac{\partial^{n} }{\partial x_d^{n}} f(x_0, x_1,\ldots, x_d) \end{align*} using the numdifftools wrappers Jacobian, Hessdiag and Hessian.

An easy fix to get high order partial derivatives from numdifftools is to modify the Jacobian by removing the restriction that n must be one. Then you can get reasonable reliable partial derivaties \begin{align*} \frac{\partial^{n} }{\partial x_0^{n}} f(x_0, x_1,\ldots, x_d), \frac{\partial^{n} }{\partial x_1^{n}} f(x_0, x_1,\ldots, x_d), \ldots, \frac{\partial^{n} }{\partial x_d^{n}} f(x_0, x_1,\ldots, x_d) \end{align*} up to n equal about 10 depending on the function you are differentiating.

pbrod avatar Apr 28 '21 23:04 pbrod

This would be a very useful enhancement.

Nickleaton avatar Jun 29 '21 13:06 Nickleaton

Looking at some bits of sklearn, there is the idea of a power used to specify terms in a multivariate polynomial

For example. (2,1,0) means x[0]^2 * x[1]^1 * x[2]^0 = x[0]^2 * x[1].

See PolynomialFeatures from sklearn.preprocessing

Passing in powers in this way could be used to specify that you want d^3X/dx^2dy for example

Nickleaton avatar Jun 29 '21 14:06 Nickleaton