Bug: Setting TRAIN.DET_ONLY to True updates encoder as well

[RahulRewale]

I wanted to fine-tune only the detection head of the model, so I set _C.TRAIN.DET_ONLY=True in the config file. However, I noticed that the encoder is also getting updated, and thus, the segmentation output is deteriorating.