ModernBERT
ModernBERT copied to clipboard
What's the difference among the different mode when initialize model from pretrained model?
def init_model_from_pretrained(
pretrained_model: FlexBertModel,
new_model: FlexBertModel,
mode: Union[str, TileMode] = TileMode.tile_weights_from_middle,
):
I notice the method take mode as an argument, and it has three kinds of mode,'center_weights', 'tile_weights_from_edge', or 'tile_weights_from_middle', may I ask what's the differences among those three, and how they influence the initializing process?