bert-squeeze icon indicating copy to clipboard operation
bert-squeeze copied to clipboard

Use Callback for 2 stage training in DeeBert

Open JulesBelveze opened this issue 2 years ago • 0 comments

DeeBert models need to be fine-tuned in a two step fashion: first the final layer and then the ramps. The current implementation requires the user to do two different training. However, this can be achieved in one-shot using a pl.Callback, as done for TheseusBert.

JulesBelveze avatar Nov 01 '22 20:11 JulesBelveze