LLM-Pruner
LLM-Pruner copied to clipboard
horseee
[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.
Thank you very much for doing such great open-source work! i try: CUDA_VISIBLE_DEVICES=X bash scripts/evaluate.sh PATH_OR_NAME_TO_BASE_MODEL PATH_TO_SAVE_TUNE_MODEL PATH_TO_PRUNE_MODEL EPOCHS_YOU_WANT_TO_EVALUATE but get the result: Selected Tasks: ['piqa', 'boolq', 'arc_challenge', 'hellaswag', 'openbookqa',...
LOGS: You are using a model of type mistral to instantiate a model of type llama. This is not supported for all configurations of models and can yield errors. Loading...
Hi, Thanks a lot for this awesome work! I am wondering whether there is a way to check the pruned but uncompressed model. Now when I save the model, they...
I’ve tried to prune Llama2-7B on a MacBook Pro M1 but the system end it by killing the process because of OOM (I’ve 32GB) Is there something I can do?...
When I'm running `python generate.py --model_type pretrain` The error occurs, I can't understand the reason...
Hi, By anychance, have you tried literally run on the llama-2 model? I tried using default llama parameters for pruning and post-training, resulting in similar wikitext2 score (~19) but much...
Could you please tell me which command you used for post training of model to get results for element 1 and element 2 method.
