sentiment neuron test

[nkooli]

hello,

i am trying to test the sentiment neuron approach in https://github.com/openai/generating-reviews-discovering-sentiment over a new dataset. I trained the model on my dataset, this generates me a three files : model.data, model.index, model.meta

How can i generates the 15 .npy files (0.npy, 1.npy, ..., 14.npy) to test the sentiment analysis code (as in in https://github.com/openai/generating-reviews-discovering-sentiment) ?

thank you !

[jonny-d]

Hello,

Sorry for the late reply!

I have added a script called extract_weights.py to generate the .npy files in the format you want. All you need to do is pass the path to your model.meta file using the --meta_path argument, and also the path to your directory containing the three files (the .data, .meta and .index files) using the --model_dir argument. This will generate a folder containing 15 .npy files (numbered from 0.npy to 14.npy) with the same order as in generating-reviews-discovering-sentiment. You can then run encoder.py from the same directory, which will load these numpy arrays for inference.

This is a bit of a roundabout way of doing it, but it works.

Hope this helps!

[nkooli]

Hello, Great ! thank you It works after changing the nbatch to 32 and nhidden to 128 in encoder.py I have to find the sentiment neuron now between [0,128] ...

[jonny-d]

To find the hidden neuron I would try to recreate figure 3 from the paper Learning to Generate Reviews and Discovering Sentiment by feeding in the positive and negative IMDB reviews and generating this plot for each hidden unit. You should do this first using the pre-trained weights supplied by the authors.

Note that if you have trained your model on new data, there is a good chance your model may not have learned to represent sentiment in the same way.

There is also this LSTMVis tool (http://lstm.seas.harvard.edu), which I have not used but looks like it could be useful.

[pickou]

I've trained mlstm network with tensorflow using part of the amazon reviews used in this paperLearning to Generate Reviews and Discovering Sentiment，but I have not found any senitment neuron. Have you ever tried to repeat the openai's work?

[jonny-d]

@pickou Sorry for the late response! Yes I have attempted to reproduce the openai work. Tensorflow code to train their model 'from scratch' is avalailabe here!