Torch-Pruning
Torch-Pruning copied to clipboard
Load error after pruning
After pruning Qwen2.5-3B using the examples/llm/prune_llm.py script and saving to get the model, I tried to load the pruned model using AutoModelForCausalLM.from_pretrained(), but it failed, reporting the following error:
The official usage documentation notes point out that the model can be loaded with AutoModelForCausalLM.from_pretrained() after pruning?
My loaded inference code is shown below:
Guys and gals, what should I do to fix this? Hope to get a reply from you all