tangent
tangent copied to clipboard
Unable to compute Hessian-vector product function if function calls other functions
Minimal test case:
import tangent
import numpy as np
def forward(theta, states):
return states
def loss(theta, states, actions):
err = forward(theta, actions)
return np.mean(err, axis=(0,))
dlossdtheta = tangent.autodiff(loss, mode='reverse')
ddlossddtheta = tangent.autodiff(dlossdtheta, mode='forward')
Fails on computation of ddlossddtheta
, with
Traceback (most recent call last):
File "scratch.py", line 12, in <module>
ddlossddtheta = tangent.autodiff(dlossdtheta, mode='forward')
[...]
File "/Users/lericson/devel/.../env/src/tangent/tangent/fence.py", line 256, in visit_IfExp
self._reject(node, 'Conditional Expressions are not supported')
File "/Users/lericson/devel/.../env/src/tangent/tangent/fence.py", line 91, in _reject
self._raise_error(msg)
File "/Users/lericson/devel/.../env/src/tangent/tangent/fence.py", line 74, in _raise_error
raise TangentParseError(msg, ('<stdin>', lineno, offset + 1, line))
File "<stdin>", line 3
axis_shape = x.shape if axis is None else tuple(x.shape[a] for a in axis)
Replacing the function call with inlining the function solves the issue.
This is running 6533e83af09de7345d1b438512679992f080dcc9
I think the state of HVP computation is actually pretty broken in both the latest release and said commit. I can't get HVPs to work for anything that isn't basically an identity function.