[QUESTION] Chapter 10 Exercise 10 Factor parameter

[AdrianVazquezTorres]

Good day, I'm having a bit of trouble understanding why the value of 1.005 was chosen for the factor parameter. I know it's supposed to be an increment by 0.5% at each iteration and that the new learning rate will be new_lr = old_lr * factor, but I don't get where that value (1.005) came from.

This is the code:

model.compile(loss="sparse_categorical_crossentropy",
              optimizer=keras.optimizers.SGD(learning_rate=1e-3),
              metrics=["accuracy"])
expon_lr = ExponentialLearningRate(factor=1.005)

• Notebook: 10_neural_nets_with_keras
• Cell: 119

[mjurzak]

I think the value of 0.5% is chosen arbitrarily just to increase from small learning rate (starting from 0.001). As was stated earlier in the chapter, the optimal learning rate is usually a half of maximum learning rate before it diverges, but we can get it from either way:

• forward (multiplying or adding)
• backward (dividing or subtracting)
  Edit: Forgot to add, that this is precisely what is done in the notebook (121th cell), the author discovers the maximum value and takes half of it (in this case 1e-3).