mlx
mlx copied to clipboard
ml-explore
MLX: An array framework for Apple silicon
Add a few learning rate schedules to `mlx.optimizers`.
Yayi 30b k,v layer has [input_dims=7168, out_dims=112], so it failed to quantize due to error `all dimensions should be divisible by 32 for now`. FYI, here is the implementation of...
**Issue Description:** ### Feature Request **Summary:** I propose adding support for parameter groups in MLX to enhance the flexibility and customization of model optimization. **Details:** The addition of parameter groups...
I wanted to contribute to the Docs with neat code snippets for Python APIs (Like snippets found in PyTorch/Jax docs). I followed the instruction in this [page](https://github.com/hwaseem04/mlx/tree/main/docs) But I am...
## Proposed changes 1. Added GLU activation function. 2. Added Gated activation function. ## Checklist Put an `x` in the boxes that apply. - [x] I have read the [CONTRIBUTING](https://github.com/ml-explore/mlx/blob/main/CONTRIBUTING.md)...
Yes, but still there's problem since you are concatenating not doing +. ```python class TransformerLM(nn.Module): def __init__(self, vocab_size: int, num_layers: int, dims: int, num_heads: int): super().__init__() self.my_encoder = transformer.MyPosEncoding(dims) self.your_encoder...
Convert to ipynb and run: https://gist.github.com/wjessup/ce49625cb551af8663059b93cbfab209 Takeaways: Smaller network torch is 3-5x faster Forward: (45.5 µs vs 235 µs) and Training: (337 µs vs. 1.22 ms) 640 x 4 layers...
Changes the computation of variance to a faster one (but slightly less numerically stable). Not sure if we want to merge this but the performance benefit for large reductions is...
`mlx.core` arrays are great already. To support better interoperability with other frameworks, even mlx.data, implementing the [buffer protocol](https://docs.python.org/3/c-api/buffer.html) as for numpy arrays would help. This should address https://github.com/ml-explore/mlx-data/issues/20 and will...
I like the statement: > MLX has a Python API that closely follows NumPy. One thing I noticed is that mx.array.shape returns a list: ``` mx.ones([2, 3]).shape [2, 3] ```...