Ability to not update part of the model

[neubig]

It would be nice to have the ability to not update part of the model. This would allow us to do something like the following paper, where they freeze some of the parameters but learn some:

https://arxiv.org/abs/1604.02201

[msperber]

I'm wondering what the cleanest way to implement this would be, requiring little or no changes to the existing model components. Preferably, we could do some stuff behind the scenes so that when a component is not to be updated, the call to ParamManager.my_params(self) would return something that behaves like a normal parameter subcollection but prevents its parameters from being updated.

One way might be to overwrite add_parameters() so that it returns an expression as W.expr(update), and then remove all calls to dy.parameter() which I think might be more conform to the current DyNet API anyways. I'm not sure if that would bring along any disadvantages, such as not being able to access some parameter-specific features?

[philip30]

One easy way to implement this is to make a wrapper with annotator that will return an expression with dy.nobackprop() if not updated else normal expression.

but this requires some dy.parameter calling to be wrapped with a method appropriately. Currently it is easy for calc_loss and the calc_nll methods.