The TF Eager example in /README.md doens't run correctly

[wangkuiyi]

Install Tangent and Eager Execution

I ran the following commands with the Ubuntu 16.04 Docker image:

• Choose version v0.1.9 in my local tangent repo.
• Start the container docker run --rm -it -v $PWD:/tangent /bin/bash where $PWD refers to my local git clone of tangent.
• Install requirements inside the containers pip install -r /tangent/requirements.txt

Run the Program

I copied the TF Eager example from /README.md and tried to run it:

import tensorflow as tf
import tangent
import numpy

tf.enable_eager_execution()
tf.executing_eagerly()

def f(W,x):
  h1 = tf.matmul(x,W)
  h2 = tf.tanh(h1)
  out = tf.reduce_sum(h2)
  return out

dfdW = tangent.grad(f, verbose=1)

x = W = [[2.]]
print dfdW(W, x)

I ran the above program by executing the following command in the container:

PYTHONPATH=/tangent python tests/a.py

The Error Messages

It sees the the autodiff works as it prints the derived dfdW, but it doesn't work running dfdW:

root@4f0441d33975:/tangent# PYTHONPATH=$PWD python tests/a.py
def dfdW(W, x, bout=1.0):
    h1 = tf.matmul(x, W)
    h2 = tf.tanh(h1)
    out = tf.reduce_sum(h2)
    assert tangent.shapes_match(out, bout
        ), 'Shape mismatch between return value (%s) and seed derivative (%s)' % (
        numpy.shape(out), numpy.shape(bout))

    # Grad of: out = tf.reduce_sum(h2)
    _bh2 = tangent.unreduce(bout, tangent.shape_as_list(h2), None, False)
    bh2 = _bh2

    # Grad of: h2 = tf.tanh(h1)
    _h2 = h2
    _bh1 = bh2 * (1 - _h2 * _h2)
    bh1 = _bh1

    # Grad of: h1 = tf.matmul(x, W)
    _bW = tangent.matmul_adjoint_y(bh1, x, W, False, False)
    bW = _bW
    return bW

2018-06-06 23:56:15.318751: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Traceback (most recent call last):
  File "tests/a.py", line 17, in <module>
    print dfdW(W, x)
  File "/tmp/tmpV9NwKl/tangent_068c.py", line 5, in dfdW
    assert tangent.shapes_match(out, bout
  File "/tangent/tangent/utils.py", line 629, in shapes_match
    shape_checker = shape_checkers[(type(a), type(b))]
KeyError: (<type 'EagerTensor'>, <type 'float'>)
root@4f0441d33975:/tangent# PYTHONPATH=$PWD python tests/a.py
2018-06-07 00:00:54.341592: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Traceback (most recent call last):
  File "tests/a.py", line 17, in <module>
    print dfdW(W, x)
  File "/tmp/tmpVEP8ir/tangent_43bb.py", line 5, in dfdW
    assert tangent.shapes_match(out, bout
  File "/tangent/tangent/utils.py", line 629, in shapes_match
    shape_checker = shape_checkers[(type(a), type(b))]
KeyError: (<type 'EagerTensor'>, <type 'float'>)

It complains that out = tf.reduce_sum(h2), which has type EagerTensor, and bout, which is of float, do not have the same type.

Could you please recommend the right way to handle this error? It doesn't seem work if I pass in an EagerTensor to bout.

Thank you!

[bbrelje]

I am getting the same issue using Tensorflow 1.10. I think the TensorFlow API changed quite a bit since version 1.5 (which Tangent is using for testing on Travis) and it broke the shape_checkers function among many others. The fact that the lead dev(s) haven't responded to this seemingly simple issue in three months is disconcerting as a potential user.

Try using Tensorflow 1.5 or if you must use a late Tensorflow, you can get derivatives at run-time using the GradientTape object

[mdanatg]

Sorry for the delay. Here's a workaround:

dfdW(W, x, bout=tf.constant(1.0))

The problem is that our shape checker doesn't know how to mix Tensor with float - it only know how to compare tensors and variables: https://github.com/google/tangent/blob/master/tangent/tf_extensions.py#L72

The fix should be fairly straightforward by adding some calls to convert primitives to tensors in tensor_shapes_match and then adding int, float, etc. to the tuple passed to register_all_shape_checker.