NeuroNER
NeuroNER copied to clipboard
Unable to restore pretrained model when using Adam optimizer
I get the following error when re-using a pretrained model, but only when I use adam as the optimizer during training/restoration:
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [13572,100] rhs shape= [53285,100] [[Node: save/Assign_47 = Assign[T=DT_FLOAT, _class=["loc:@token_embedding/token_embedding_weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](token_embedding/token_embedding_weights/Adam_1, save/RestoreV2_47)]]
The error is generated during execution of line 337 in entity_lstm:
self.saver.restore(sess, pretrained_model_checkpoint_filepath) # Works only when the dimensions of tensor variables are matched.
Everything goes smoothly on the exact same data set when sgd is the optimizer.
UPDATE: I found a workaround for this -- just change the optimizer to sgd in the parameters file when re-using the model. (Note, in this scenario I'm just deploying the model, not continuing to train it, so the optimizer isn't getting used).