robotsp

@gmryu Does it mean nni does not support fairseq framework yet?

> @robotsp sorry, I cannot reply you during my work time. (Also, have you actually read all I wrote? Is my phrasing too bad to understand?) > > Nonetheless, you...

@gmryu I have checked some weight freezing methods, but all of them are to freeze a whole module rather than some tensors (in my case, I only want to freeze...

> (or you may try to freeze some tensors,i.e. model weights, and reload again. It will identifies whether saving tensors keeps their frozen state or not.) It is similar to...

@gmryu , I found that params.requires_grad=False isn't working for fairseq model to freeze weights? 1. I used torch.load("checkpoint.pt"), and reset the params.requires_grad=False. 2. I saved it as a weights-frozen model...

@gmryu Thanks. I have a try then.

@gmryu I have adjusted the code in fairseq_cli/train.py, I defined a pruning function: ``` def prune_model_global_unstructured(model, layer_type, proportion): from torch.nn.utils import prune import torch.nn as nn module_tups = [] for...

> @gmryu I have adjusted the code in fairseq_cli/train.py, I defined a pruning function: > > ``` > def prune_model_global_unstructured(model, layer_type, proportion): > from torch.nn.utils import prune > import torch.nn...

@gmryu you are so welcome.

> I used the method from this issue, it worked for me [#2120 (comment)](https://github.com/facebookresearch/fairseq/issues/2120#issuecomment-647429120) > I have similar question. Have you found a suitable method? prune or distill. Not found...