MSVA

MSVA copied to clipboard

Issue regarding the last step of self attention (weighted sum step)

3

Hi, I noticed that the last step of the self-attention calculation doesn't seem so right: ``` att_weights_ = nn.functional.softmax(logits, dim=-1) weights = self.dropout(att_weights_) y = torch.matmul(V.transpose(1,0), weights).transpose(1,0) ``` So here...

pangzss

sunguoquan1005

can't reproduce the f1 results, and coefficient results also seems unusual

3

ZiyiZhang27

Extracted features for the datasets

1

could you help me with how to extract features of the datasets? ( I also want to train in my dataset demo)

johnbager