stanza icon indicating copy to clipboard operation
stanza copied to clipboard

Published 20 hours ago • verified publisher icon stanfordnlp

Bias not used

Open kmkurn opened this issue 4 years ago • 3 comments

Hi, it seems this bias weight in PairwiseBilinear is not used. Possibly a (minor) bug? https://github.com/stanfordnlp/stanza/blob/1c73418b65905b2d32f7b0dcb0bdd4a615b6ab0a/stanza/models/common/biaffine.py#L17

kmkurn avatar May 13 '20 06:05 kmkurn

Thank you for pointing out! This is indeed a bug that we confirmed. However, we suppose this small bug might have little influence on model performance. We will figure it out and fix this in future release!

yuhui-zh15 avatar May 25 '20 07:05 yuhui-zh15

https://github.com/stanfordnlp/stanza/blob/e6c0b338ec7a5fdff422cb275ef7c448003e95f6/stanza/models/common/biaffine.py#L24-L29

Besides, I don't understand why intermediate tensor can view from (N x L1) x (D2 x O) to (N x (L1 x O) x D2) without transpose.

Hu-chi avatar Aug 13 '20 02:08 Hu-chi

If I go to this line: https://github.com/stanfordnlp/stanza/blob/1c73418b65905b2d32f7b0dcb0bdd4a615b6ab0a/stanza/models/common/biaffine.py#L32

and add the bias, such as with

output = output + self.bias

I then retrained on a few datasets. One model got slightly better, one model stayed the same, and four got slightly worse. My conclusion is that the bias value isn't actually useful in this model.

AngledLuffa avatar Apr 25 '22 03:04 AngledLuffa