pytorch-sgns

pytorch-sgns copied to clipboard

Confused by the loss function.

2

In your code, you minimized `-(oloss + nloss).mean()` which means (oloss+nloss) should be large. So, "oloss become large and nloss become small " is expected. Although -(oloss+nloss) decrease, I got...

JinYang88

torch tqdm

rjpruitt16

Where is Expectation

5

 In this formula we have an expectation of $w_i$. That means for each pair of $(w_I, w_O)$ we should calculate this expectation. But as I can...

zetyquickly

why initialize the first line of embedding matrix as 0? t.zeros(1, self.embedding_size)

5

yl-jiang

myConfusion

1

Why code below, in the project, can be used as "loss". ` oloss = t.bmm(ovectors, ivectors).squeeze().sigmoid().log().mean(1) nloss = t.bmm(nvectors, ivectors).squeeze().sigmoid().log().view(-1, context_size, self.n_negs).sum(2).mean(1)` In my judgment, "loss" should be "prediction" -...

tutou-pifeng

Purpose of unks in skipgram function

2

Hi, Can you please explain, what is the purpose of including the `` tokens in the `owords` vector produced by `skipgram` function? What should model learn by using these as...

mmlynarik