fairseq
fairseq copied to clipboard
facebookresearch
Is there any method can be used to prune translation model in fairseq?
❓ Questions and Help
Before asking:
- search the issues.
- search the docs.
What is your question?
Code
What have you tried?
What's your environment?
- fairseq Version (e.g., 1.0 or main):
- PyTorch Version (e.g., 1.0)
- OS (e.g., Linux):
- How you installed fairseq (
pip, source): - Build command you used (if compiling from source):
- Python version:
- CUDA/cuDNN version:
- GPU models and configuration:
- Any other relevant information:
I have similar question. Have you found a suitable method? prune or distill.
I used the method from this issue, it worked for me https://github.com/facebookresearch/fairseq/issues/2120#issuecomment-647429120
I used the method from this issue, it worked for me #2120 (comment)
I have similar question. Have you found a suitable method? prune or distill.
Not found yet..
I used the method from this issue, it worked for me #2120 (comment)
@tianshuailu Cool, thanks! Are you familiar with microsoft nni ? Can we use it to prune fairseq model ?
I used the method from this issue, it worked for me #2120 (comment)
@tianshuailu Cool, thanks! Are you familiar with microsoft nni ? Can we use it to prune fairseq model ?
Sorry, I'm not familiar with Microsoft nni, so not sure if it works
I have similar question. Have you found a suitable method? prune or distill.
@tianshuailu Some pruning methods are introduced in torch. Not sure if it helps. https://pytorch.org/tutorials/intermediate/pruning_tutorial.html
@tianshuailu hi tianshuai, do you know how to freeze parameters during finetuning in fairseq? I used both zero_grad and requires_grad to false are not working in fairseq train.
@tianshuailu hi tianshuai, do you know how to freeze parameters during finetuning in fairseq? I used both zero_grad and requires_grad to false are not working in fairseq train.
Sorry I haven't tried to freeze the parameters for our project, so I dont know much about it.