wevi
wevi copied to clipboard
Derivative of softmax and consistent subscripts to paper
Hi @ronxin, Thank you for your awesome paper and wevi, which visualise the word2vec in a feasibly easy way.
Think it would be interesting to contribute to wevi while learning. After reading your codes I found the back-propagation function a bit strange. Seems that you didn't take into account the derivative of the softmax, aka the final output values, with respect to the values before taking softmax. Wondering is this some sort of approximation?
In this pull request, I have coded the derivative into the back-propagation function. In addition, I modified the subscripts as in the paper, i.e., k for input neurons, etc.
Please have a look if they are all good.
Didn't really do testing. Only run a couple of examples, looks like the king, queen, man, woman relation still holds.