Chapter 11: code formatting is broken

[christophboesch]

Code formatting in Chapter 11, section "Predicting Movie Reviews" is broken. I was not able to reproduce the results in the book, p. 196, and assume that there is some error in my loop structure. Could could you update the formatting of the example?

[naruto678]

Could you provide the code for better clarification ? @christophboesch

[christophboesch]

Could you provide the code for better clarification ? @christophboesch

Sure! And thank you for the fast response!

import numpy as np
np.random.seed(1)

def sigmoid(x):
    return 1/(1 + np.exp(-x))

alpha, iterations = (0.01, 2)
hidden_size = 100

weights_0_1 = 0.2 * np.random.random((len(vocab),hidden_size)) - 0.1
weights_1_2 = 0.2 * np.random.random((hidden_size,1)) - 0.1

correct,total = (0,0)

for iter in range(iterations):
    for i in range(len(input_dataset)-1000):
        x,y = (input_dataset[i],target_dataset[i])
        layer_1 = sigmoid(np.sum(weights_0_1[x],axis=0))
        layer_2 = sigmoid(np.dot(layer_1,weights_1_2))
        
        layer_2_delta = layer_2 - y
        layer_1_delta = layer_2_delta.dot(weights_1_2.T)
        
        weights_0_1[x] -= layer_1_delta * alpha
        
        if(np.abs(layer_2_delta) < 0.5):
            correct += 1
        total += 1
        if(i%10 == 9):
            progress = str(i/float(len(input_dataset)))
            sys.stdout.write('\rIter:'+str(iter)\
                             +' Progress:'+progress[2:4]\
                             +'.'+progress[4:6]\
                             +'% Training Accuracy:'\
                             +str(correct/float(total)) + '%')
    print()
correct,total = (0,0)
for i in range(len(input_dataset)-1000,len(input_dataset)):
    
    x = input_dataset[i]
    y = target_dataset[i]
    
    layer_1 = sigmoid(np.sum(weights_0_1[x],axis=0))
    layer_2 = sigmoid(np.dot(layer_1,weights_1_2))
    
    if(np.abs(layer_2 - y) < 0.5):
        correct += 1
    total += 1
print("Test Accuracy: " + str(correct / float(total)))

[naruto678]

Found the reason ... you are not updating the weights_1_2 for the entire training and hence you are not able to train the model.

[christophboesch]

@naruto678: Thank you! That was exactly the problem. @iamtrask: Maybe it is still possible to fix the formatting in the code example to facilitate self diagnostics of errors ;)

[naruto678]

What do you mean when you say correction of formatting???

[christophboesch]

The coding example for chapter 11, section "Predicting Movie Reviews" looks like this: If the second part of the code would also make use of indentation and syntax highlighting, it would be easier to compare the solutions.

[naruto678]

That is true .I will try to correct the formatting but it is up to @iamtrask to accept the pull request.

[christophboesch]

Actually, now that you mention it, I can create the pull request myself. With your help I have a working version of the code now ;) Thanks once again.