[Q] : why Softmax function take computed only on abs values and ignore imaginary part

[deep-matter]

i tried to re-implemented on both of them following this code :+1:

import torch
from torch.nn import functional as F


def complex_softmax(input, dim=None):
    """Applies the complex softmax function to an input tensor.
    """
    B , C , Freq,time_d = input.size()

    real = F.softmax(input.real, dim=dim)
    imaginary = F.softmax(input.imag, dim=dim)
    magnitudes = torch.sqrt((real ** 2 + imaginary ** 2))
    return (real * magnitudes).view(B*C ,Freq,time_d) * (
        imaginary * magnitudes
    ).view(B,C ,Freq,time_d)

[omrijsharon]

The imaginary part is not ignored. This is the implementation of the abs method of a torchlex tensor: def __abs__(self): return self.real**2 + self.imag**2
and the softmax uses it: def softmax(z, dim): ''' Complex-valued Neural Networks with Non-parametric Activation Functions (Eq. 36) https://arxiv.org/pdf/1802.08026.pdf ''' if "ComplexTensor" in z.__class__.__name__: result = torch.softmax(abs(z), dim=dim) else: result = torch.softmax(z, dim=dim) return result
as you can see, even with the relu function: there are many ways to implement the regular activation functions on a complex tensor. all i have done is implement what i found in the literature, in this case here: Complex-valued Neural Networks with Non-parametric Activation Functions (Eq. 36) https://arxiv.org/pdf/1802.08026.pdf
you can also use the pdf method which mimics the way you get position probabilities from the wave-function in quantum mechanics. the pdf also outputs a probability.

you are correct that there was no consideration of the dim and that needs to be fix. as you know, there are now complex tensors buildin in pytorch so a new library contains complex functions on the pytorch complex tensors is needed.

thank you for the fix!