Angus Graham

Results 1 issues of Angus Graham

Seems the default mlx weight initialization is incorrect. Specifically github mlx/python/mlx/nn/layers/linear.py and other similar references. Generally the starting point for weight init is a normal distribution with mean = zero...