Add dimension-wise reverse function and generalized transpose function

[vbkaisetsu]

Add:

• dimension-wise reverse function
• generalized transpose function

For example:

/*
  A = {
     1, 2, 3, 4, 5,
     6, 7, 8, 9,10,
    11,12,13,14,15,
    16,17,18,19,20,
  };

  Functions:
    reverse(x, dim);
    reverse(x, dims);
    transpose(x, dim1, dim2);
 */

B = functions::reverse(A, 0);
/*
  B = {
    16,17,18,19,20,
    11,12,13,14,15,
     6, 7, 8, 9,10,
     1, 2, 3, 4, 5,
  };
 */
C = functions::reverse(A, 1);
/*
  C = {
     5, 4, 3, 2, 1,
    10, 9, 8, 7, 6,
    15,14,13,12,11,
    20,19,18,17,16,
  };
 */
D = functions::reverse(A, {0, 1}); // or functions::reverse(A, 0, 1);
/*
  D = {
    20,19,18,17,16,
    15,14,13,12,11,
    10, 9, 8, 7, 6,
     5, 4, 3, 2, 1,
  };
 */
E = functions::transpose(A, 0, 1);
/*
  E = {
     1, 6,11,16,
     2, 7,12,17,
     3, 8,13,18,
     4, 9,14,19,
     5,10,15,20
  };
 */

[odashi]

transpose is useful, but I didn't recognize a high necessity of reverse. Please add some use-cases to this thread.

[vbkaisetsu]

It is mainly for image processings using conv2d.

Use case 1: Reverts a kernel before conv2d

w = F::reverse(F::reverse(kernel, 0), 1);
y = F::conv2d(x, w, 0, 0, 1, 1, 1, 1);

Use case 2: Creates a symmetric kernel

w0 = kernel;
w1 = F::reverse(w0, 0);
w2 = F::reverse(w0, 1);
w3 = F::reverse(w1, 1);
w = F::concat(F::concat({w0, w1}, 0), F::concat({w2, w3}, 0), 1);

[odashi]

Any existing studies?

[odashi]

NumPy has a function flip that behaves similarly to reverse: https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.flip.html and swapaxes similarly to transpose: https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.swapaxes.html

Especially, NumPy's transpose has different semantics with above: https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.transpose.html

Function names should be matched to reduce user's confusion.

[vbkaisetsu]

Tensorflow has reverse and transpose functions. https://www.tensorflow.org/api_docs/python/tf/reverse https://www.tensorflow.org/api_docs/python/tf/transpose transpose takes perm argument that means the new order of axes.

Chainer has flip and swapaxes functions. http://docs.chainer.org/en/stable/reference/generated/chainer.functions.flip.html http://docs.chainer.org/en/stable/reference/generated/chainer.functions.swapaxes.html

DyNet's transpose function takes two dimensions instead of a permutation. http://dynet.readthedocs.io/en/latest/operations.html

NumPy	Chainer	Tensorflow	DyNet
flip	flip	reverse	N/A
swapaxes	swapaxes	transpose	transpose

[odashi]

Following names in NumPy is recomnended to reduce user's confusion. TF's reverse is floating its specification due to new release and we should not follow it for now. DyNet functions sometimes do not follow commonly used names.

[odashi]

Would you like to change the name of our function reverse to flip?

[vbkaisetsu]

Okay. I changed the name of function to flip(), and pushed it.