Iz Beltagy

Thanks, @dlibenzi.

@dlibenzi, sorry, I am not sure I am following how this link is related to unfold

I see. This is the c++ tensorflow version of `torch.unfold`. But this is not something that can be called from pytorch-xla?

hey @JackCaoG, I am just curious if there are updates here.

Thanks, @JackCaoG for the forward function in your PR [here](https://github.com/pytorch/xla/pull/2326). I ran your code and I successfully get `Counter: xla::unfold` and still get `Counter: aten::unfold_backward` as expected. There are a...

> Did you remember how much time it takes prior to the unfold change? around 5 minutes > [12 * 2048 * 64 - 512, 1, 12 * 2048 *...

If it is possible to implement `unfold` as a view, that would be the ideal solution because it won't waste any memory, which is the bottleneck in the Longformer model.

Will try both and let you know. Thanks.

I tried the iterative slicing that you suggested and found it to work well. The memory usage is low enough that I can run the model on long sequences, and...

Sure. I will move the model optimization to a separate issue. One thing that's still relevant here is finding out if `unfold` can be lowered as a view without additional...