Document difference between foo_out and foo_ (inplace)

[ezyang]

My understanding of the situation:

• A foo_out function will perform a resize on the output tensor, and then write data into it.

• A foo_ function will NOT perform a resize and write data directly.

TBH, I am not really sure why we ever actually need foo_out, if you have the variant that allocates and returns a new tensor immediately.

[gchanan]

foo_ treats the output as an input, whereas foo_out doesn't.

[ezyang]

What does it mean to "treat as an input"?

[gchanan]

I mean it's not an input to the actual function being computed (e.g. add):

Type.add_(x, y) is roughly (ignoring things like resizing and inplace): x.set_(x+y)

while Type.add_out(z, x, y) is roughly equivalent to: z.set_(x+y)

so it just allows you to use an external "buffer" to store the result. I don't know if I've seen a case where that's useful though.

[ezyang]

Suppose I have the function def f(x, y): x.set_(x * 0 + y). Is this an inplace function or an out function? Semantically, this is equivalent to x.set_(y) (e.g., an out function), but I did "use" x as an input (I simply multiplied it by zero.) So, while I agree with the intuition of "was it used as an input to the function", this intuition seems to work poorly in cases like this. This is not a "fake" example, because the CuDNN APIs let you swap between "overwrite" and "accumulate" by setting a multiplier on x, even though we always set this multiplier to zero. So you end up in the funny situation where, x is not used as an input at all, but it really is an input, at least as far as the implementation goes.

I thought maybe resizing and broadcasting would add more clarity to the matter, but it seems that, no matter if we are running a function as inplace or as out, we always call resize on the output tensor to get it into the same shape as the "self" argument. This, combined with the really weird behavior of resize (c.f. https://github.com/pytorch/pytorch/issues/1570) means that actually making use of _out variants is quite hazardous (you better not pass a view with the wrong size, it will get clobbered! Fortunately, we don't expose _out variants to end users.)