Representation of similar words

[pberko]

Hello

hope the question is in the right place

I used kind of https://github.com/bentrevett/pytorch-sentiment-analysis/blob/master/5%20-%20Multi-class%20Sentiment%20Analysis.ipynb for prediction of unique regular language.

The results where realy good!

I want to check if prediction of similar words "turn on" the same neurons.

Is there a way to check that?

I thaught that if I'll have a representation of the words (vector) and the vector will be similar - it will prove it.

Am I right? how can I represent this vector?

Thanks

[bentrevett]

I'm not 100% sure of the best way of doing this. One way would to have your model simply output the tensors from the intermediate layers, i.e. return self.fc(cat), cat, pooled, conved, embedded and then compare them for each example.

A cleaner way would be to use "hooks", see: https://pytorch.org/tutorials/beginner/former_torchies/nnft_tutorial.html#forward-and-backward-function-hooks. Although I don't have much experience with these.