Guorun

Hi, did you successfully run the train.ipynb? @subzerofun I encountered this problem! Did you? Thanks!

Hi, so `diag` is a diagonal matrix with the `zint` on its diagonal line？

Thanks! So why it is have to be a diagonal matrix? Can a non-diagonal matrix replace it as long as the non-diagonal matrix represents the corresponding mask?

Hi @xiamengzhou thanks for your kind reply. In your code, do you use `prune_head` to directly cut the whole `Query` and use `prune_layer` to cut the element in the `Query`...

Sorry to bother again, may I ask that how do you prune the model during the process. I see in this line you obtain the `zs`. https://github.com/princeton-nlp/CoFiPruning/blob/main/trainer/trainer.py#L282 But then how...

您这里标注的是friend呀？@Huxpro

Hi @cli99 , I have written a PR a few weeks ago, it probably work the same. https://github.com/microsoft/DeepSpeed/pull/2515

> @microsoft-github-policy-service agree

Hi @tjruwase , do you mean the `.DS_Store`? I have deleted it.

Hi @cli99 I have written a PR a few weeks ago, they probably could merge together. https://github.com/microsoft/DeepSpeed/pull/2515