luwei

Questions about calculate_weights()

2

Hi Andrey, I have a couple of questions about calculate_weights(). 1. in line 106-109, why do you need expand_as(self.y)? 2. in line 118, why do you need to divide g...

dropout ratio for inputs

1

In the layer.py line 170, there is `x = tf.nn.dropout(x, 1-self.dropout)`. By the documentation of tf, the argument for `tf.nn.dropout` is the ratio to drop. So should it be `x...

conv1d with bias=False

2

Summary: Under the same setting as the last diff, we support `bias=false`. Differential Revision: D56285842

conv1d

1

Summary: We follow D50914117 to implement a specific case of conv1d for our needs. Specifically, we require - the input tensor to have a single batch - groups == in_channels...

optimized matrix multiplication

3

Summary: We port the [efficient `mm` algorithm](https://github.com/pytorch/pytorch/pull/112918?fbclid=IwAR2wFuh--eEEkrFjG9qPWNE8wVU5hO9e7824XdWC1BdZMAk3f8xT8EpyhHc) by liuk22 and yipjustin into ExecuTorch. Both input matrices are assumed to be channel-packed and are repacked into with-packed and height-packed respectively. Differential...

register `view`, `reshape` and `select`

1

Summary: - We register `select`, `unsqueeze` and `view` in `vulkan_partitioner.py` in order to run vulkan_delegate test (Python e2e test). The latter two might be used to implement `bmm` and `addmm`,...

consolidate shader files `sum_dim` and `sum_dim_keepdim`

2

Summary: We consolidate the shader files `sum_dim` and `sum_dim_keepdim` into one. Reviewed By: jorgep31415 Differential Revision: D55714879

softmax

3

Summary: tsia Differential Revision: D56686859

log_softmax

6

Summary: Use the following property to implement `log_softmax` ``` log(exp(x_i) / sum(exp(x_j))) = x_i - log(sum(exp(x_j))) ``` - numerically stable, since we can avoid log(small_number) -> -\infty - reusing existing...

`bmm` and `baddbmm`

1

Summary: We implement `bmm`. Thereafter `baddbmm` is also supported since it is [decomposed as](https://www.internalfb.com/code/fbsource/[f7ead67da83314531c85723ab852b61e9d9abdd0]/xplat/caffe2/torch/_decomp/decompositions.py?lines=4680-4694) ``` aten.add.Tensor aten.mul.Tensor aten.mul.Tensor aten.bmm.default ``` Differential Revision: D56679087