Awni Hannun
Awni Hannun
Was the model actually training? Can you add the log here?
The format of the text doesn't really matter. What matters more is that you are consistent in your dataset. So if you use a special keyword in the training set,...
What model are you training? It looks like you ran out of memory... By the log, I just meant the output of the `python lora.py ...`.
Thanks, looks like it is training very very slowly (the loss didn't change much over the 300+ iterations), not sure why. The train loss should go down faster. Were you...
> But in train.jsonl, there is a line that's very long, like 2000 words Is that the default `train.jsonl` or a custom one? You should split those lines o/w they...
> By split them, do you mean split them into separate training data records? i.e. Like you have it in the first case. Split them into different lines in the...
> Oh fudge should I have put a comma in between each record? No actually, I think it will break if they have commas.
That would be awesome! For conversions usually the best place to start is a reference PyTorch implementation. This is a slightly bigger project since it involves multiple models (encodec, the...
Is anyone working on a port of Encodec? If not I might take a stab myself as I'm interested in getting some audio generation up and running!
Got it. Ok let me know what's missing in terms of layers etc that should be in `mlx.nn` and we can prioritize getting them in. For example there is a...