Implementation of Dirac Initialization Method

Open Saanidhyavats opened this issue 1 year ago • 3 comments

I was thinking to add https://pytorch.org/docs/stable/modules/torch/nn/init.html#dirac initialization method. It can be a useful feature for MLX.

Oct 19 '24 04:10 Saanidhyavats

What do you think about it @awni ?

Oct 21 '24 19:10 Saanidhyavats

It's a bit niche.. so I'm not sure it's worth adding. I see that it's in PyTorch..but I also don't know who uses that anymore. What we can do is leave the issue open.. and if it gets attention we can consider adding it reactively. I think it's better than adding code that will never / rarely be used.

Oct 22 '24 02:10 awni

Okay, sounds good

Oct 22 '24 04:10 Saanidhyavats