ModernBERT icon indicating copy to clipboard operation
ModernBERT copied to clipboard

What's the difference among the different mode when initialize model from pretrained model?

Open Lynnzake opened this issue 11 months ago • 0 comments

def init_model_from_pretrained(
    pretrained_model: FlexBertModel,
    new_model: FlexBertModel,
    mode: Union[str, TileMode] = TileMode.tile_weights_from_middle,
):

I notice the method take mode as an argument, and it has three kinds of mode,'center_weights', 'tile_weights_from_edge', or 'tile_weights_from_middle', may I ask what's the differences among those three, and how they influence the initializing process?

Lynnzake avatar Jan 03 '25 08:01 Lynnzake