Error while merging

[AmeenAli]

Hello

thanks for sharing the code!

I am using the weights from https://huggingface.co/hzf666/TSPO-0.4B

when i run python scripts/merge_weights.py

i get

selector.load_state_dict(module_state_dict)

File "/home/BIMBA/.venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for MultiModal_Align: Missing key(s) in state_dict: "temporal.Self_q.weight", "temporal.Self_q.bias", "temporal.Self_k.weight", "temporal.Self_k.bias", "temporal.Self_v.weight", "temporal.Self_v.bias", "temporal.ffn_o.weight", "temporal.ffn_o.bias", "mlp.0.weight", "mlp.0.bias", "mlp.2.weight", "mlp.2.bias".

[Hui-design]

Hi, are you performing inference or training? If you are only using the weights for inference, you don't need to run scripts/merge_weights.py — that script is only required if you're training your own model.