How to load rembert distilled models?

[kaliaanup]

Hi I am trying to load rembert distilled models for some of my downstream tasks. However, I am not able to do so.

AutoTokenizer.from_pretrained(model, **kwargs)

Can you help?

[kaliaanup]

I tried the following

def convert_tf_checkpoint_to_pytorch(tf_checkpoint_path, bert_config_file, pytorch_dump_path):
    # Initialise PyTorch model
    config = RemBertConfig.from_json_file(bert_config_file)
    print(f"Building PyTorch model from configuration: {config}")
    model = RemBertModel(config)

    # Load weights from tf checkpoint
    load_tf_weights_in_rembert(model, config, tf_checkpoint_path)

    # Save pytorch-model
    print(f"Save PyTorch model to {pytorch_dump_path}")
    #torch.save(model.state_dict(), pytorch_dump_path)

However its giving the error File "/path/miniforge3/lib/python3.9/site-packages/transformers/models/rembert/modeling_rembert.py", line 139, in load_tf_weights_in_rembert raise ValueError(f"Pointer shape {pointer.shape} and array shape {array.shape} mismatched") ValueError: Pointer shape torch.Size([256]) and array shape (128,) mismatched