How to train?

[TianshengSun]

Hi, Thanks for this implementation. But I still have some problems. The parameters in model are initialize as follow：

  def _init_weights(self, m):
    if isinstance(m, nn.Linear):
      with torch.no_grad():
        trunc_normal_(m.weight, std=.02)
      if isinstance(m, nn.Linear) and m.bias is not None:
        nn.init.constant_(m.bias, 0)
    elif isinstance(m, nn.LayerNorm):
      nn.init.constant_(m.bias, 0)
      nn.init.constant_(m.weight, 1.0)

Does this mean I should make these parameters trainable when I train this model on my own dataset?