Awni Hannun
Awni Hannun
This is fixed.
Not a priority at the moment, but definitely welcome contributions there. I'll leave this issue open to see if others are interested a Go front-end.
I'm definitely on board with that. I'm not sure exactly where we would put it (presumably a PyPi dist would be useful?). Will give it some thought.
Yea that's missing in our example. I think it would be nice to have an option on the LoRA layer which merges the adapters and the linear weights after the...
You needn't worry about this being inefficient: ``` self.linear.weight += (self.lora_a @ self.lora_b).T * 2.0 new_linear = nn.Linear(input_dims, output_dims, bias=False) new_linear.weight = self.linear.weight ``` The `new_linear.weight` is not doing a...
> I noticed that memory usage increased to around 100GB during the merging process, so I thought it might be a deep copy issue. Wow! That's a lot. It could...
> So I am proposing to move from np.savez to mx.savez in all examples, where applicable and try to keep the original dtype of the models unless explicit conversion like...
Are you interested in making this change @dastrobu ? (Our llm example surface area is getting large, so we could also wait on this until we do a bit of...
> @dastrobu Not following; it looks like you are converting the bfloat16 weights to float32, and saving them as float32. At what point do you convert them back to bfloat16?...
All great points! Thanks! I'll take a look at your PR for this