John Schulman

Unfortunately there's no good reference that covers the graph-based autodiff used in modern software. I recommend the book Evaluating Derivatives by Griewank, which covers reverse-mode autodiff. Also read up on...

That's because of how `prod` is implemented (and this is a lazy implementation, which should be fixed) ``` python def prod(x, axis=None, keepdims=False): """ Like numpy.prod """ return cgt.exp(cgt.sum(cgt.log(x), axis=axis,...

Nice work -- I'll review it in the next day or two.

Right, this functionality currently isn't implemented. It'll certainly be possible to implement a Scan-like Op (in fact, I've implemented something similar before). But I think it'll take some thought to...

Not yet, because of how the graph execution works. I'm going to open up an issue for further discussion on this topic -- @hojonathanho and I have discussed it a...

Agreed, will fix.

Jonathan, I like the idea of having cross-language serialization using some standard format. One constraint is that we'll need to serialize numerical array data. This would be kludgy in JSON....

The ReturnByRef and ReturnByVal instructions have to store the closure data for the Ops they're associated with. And a Constant Op has a value associated with it, so it'll be...

Agreed that that it makes sense to serialize through in C++. But the closure data for the Ops is created in python. How do you plan to serialize each piece...

It's bad that it fails without throwing an exception.