qml icon indicating copy to clipboard operation
qml copied to clipboard

[BUG] TensorFlow version of the backprop tutorial is producing incorrect benchmarks

Open josh146 opened this issue 4 years ago • 3 comments

The time scaling plot in the backprop tutorial is currently incorrect:

image

Strangely enough, the first section of the demo, comparing single forwards and backwards pass times for parameter-shift vs. backprop, seems to still be returning sensible times.

josh146 avatar Apr 20 '21 17:04 josh146

Update: this looks to be a TensorFlow specific issue, see https://github.com/PennyLaneAI/pennylane/issues/1244. Swapping this demo over to Autograd reverts to the previous timing behaviour.

josh146 avatar Apr 21 '21 15:04 josh146

More info:

The last build in which this demo was correctly passing occurred 8 days ago with build 2968, with demo artifact available here.

For longevity, this build had the following output:

Forward pass (best of 3): 0.005797430699749384 sec per loop
Gradient computation (best of 3): 2.5100458409004203 sec per loop
>>> print(2 * forward_time * params.size)
2.0870750519097783

image image

josh146 avatar Apr 23 '21 10:04 josh146

Update: the demo has been swapped to use autograd, while we investigate this issue.

josh146 avatar Apr 23 '21 10:04 josh146